The UK Government should take a look at Clojure

Today saw the launch of a UK Government White Paper on Open Data.

In his forward, Francis Maude writes :-

“Data is the 21st century’s new raw material…”

At EuroClojure this year I spoke about the numerous strengths of Clojure as an
enabling technology to release trapped data and onto the web. I think
this is one of Clojure’s best kept secrets.

Then there are examples of Clojure’s strengths in processing data in
order to find new insights.

And at last Tuesday’s London Clojure Dojo I witnessed a powerful example
of how Clojure can consume and leverage data on the web.

Perhaps to mark the centenary of Alan Turing’s birth, 4 fellow
Clojurians chose to create a chatbot that could engage humans in
conversation. In the space of an hour-and-a-half they had created a demo
from nothing. The code, written using LightTable, showed how they had
‘cheated’ by using the user’s input to drive a search against Twitter,
processing the JSON results and printing back a response. (I, for one,
was totally taken in at first however).

The impressive thing for me was not the ‘fake AI’ but the speed at which four
developers could craft something that sourced data from the web and
manipulated it in such an effortless, almost nonchalent, manner.

I commented after the demonstration that I strongly felt this is how
programming will be in 10 years time. Future programmers just won’t
understand why things were seemingly so difficult for us today – the
industry is awash with chatter about efficient serialization formats,
parsers, DSLs, middleware, integration technologies – and yet these guys
made it all look so easy. Once you’ve mastered Clojure, of course, it
is.

Notes on ‘rethinking web services’

I recently read a good article by Matt Aimonetti on creating web services. He makes some particular good points.

Web APIs are increasingly exposing raw data

“But web APIs are also more and more used to expose raw data to other software. The challenge though, is that we haven’t changed the way we write web applications to adapt to this new way of providing data.

Lately more and more applications provide their raw data to the outside world via web APIs. ”

Matt observes that developers are increasingly focussed on the data. This could be for the reason that Open Data is becoming more established (eg. http://data.gov.uk) or perhaps because such web APIs are better designed this way and a selection mechanism is driving the surviving population of APIs on the web.

I agree too our current tools and methods could be out of date. Perhaps even our language needs updating: how accurate is it call these things ‘web APIs’ or ‘web services’?

Documentation

Matt makes another excellent point about documentation. Users of this exposed data may not have access to or understanding of your source code. They may not even be developers. So explicit documentation is important, and documentation is invariably more accurate if generated.

“For that I use a Domain Specific Language (DSL) which is a fancy way to say that I have some specific code allowing me to explicitly define my web services. These services exist as objects that can then be used to process requests but also to validate inputs, and even more importantly to generate documentation.”

Here’s an example of his DSL, which I’ve combined with the code for this implementation.

describe_service "hello_world" do |service|
  service.formats   :json
  service.http_verb :get
  service.disable_auth # on by default

  # INPUT
  service.param.string  :name, :default => 'World', :doc => "The name of the person to greet."

  # OUTPUT
  service.response do |response|
    response.object do |obj|
      obj.string :message, :doc => "The greeting message sent back. Defaults to 'World'"
      obj.datetime :at, :doc => "The timestamp of when the message was dispatched"
    end
  end

  # DOCUMENTATION
  service.documentation do |doc|
    doc.overall "This service provides a simple hello world implementation example."
    doc.example "<code>curl -I 'http://localhost:9292/hello_world?name=Matt'</code>"
  end

  # SERVICE IMPLEMENTATION
  # the returned value is used as the response body, all the Sinatra helpers
  # are available.
  service.implementation do
    {:message => "Hello #{params[:name]}", :at => Time.now}.to_json
  end
end    

What struck me about this is how similar it is to Philipp Meier’s compojure-rest for Clojure. For example, I’ve translated his example into compojure-rest’s defresource syntax :-

(defresource 

  ^{:doc "This service provides a simple hello world implementation example."
    :example "<code>curl -I 'http://localhost:9292/hello_world?name=Malcolm'</code>"
    :params ["name" "The name of the person to greet."]} 

  hello-world

  :available-media-types ["application/json"]
  :method-allowed? (request-method-in :get)
  :authorized? true

  :handle-ok (fn [context]
               {:message (format "Hello %s" 
                           (or (get-in context [:request :params "name"]) 
                               "World")) ; default
                :datetime (java.util.Date.)}))

In compojure-rest, we should be able to add test metadata to the defresource which, along with the map declaring the service itself, could be used to generate documentation, examples and tests.