14Mar/120

A Core.logic Primer

I've written a primer, with help from Dave Nolen (many thanks !) on using core.logic which is up on the github project page. Go take a look: bit.ly/ymYvrd and let me know what you think.

Tagged as: Clojure No Comments

29Aug/115

Composition of SQL Oueries (and a real use case for a macro)

Introduction

In SICP the authors stress composition frequently, and like all abstract slipperiness I'm still working on understanding and applying consistently in my code. I recently ran across a great example of composition while writing some code to perform SQL operations, and so I'm posting about it. I will show to interact with a SQL database using queries that are composable, and that don't leave an open database connection lying around. It also demonstrates a genuine use case for a macro in order to do so. The final code is available on GitHub. I hope you enjoy.

Basic Ideas

The new contrib lib for interacting with SQL is java.jdbc. To create a function which inserts a map into a database, where the keys of the map match the field names of the database table you can do something along these lines

(ns demo
  (:require [clojure.java.jdbc :as sql]))

;; Setup the routing to the database
(def db {:classname "com.mysql.jdbc.Driver"
         :subprotocol "mysql"
         :subname "//dbserver:3306/dbname"
         :user "demo"
         :password "demo"})

(defn ins
  "Inserts a map into the database. The keys of the map must map onto the fields
   of the database.  In most cases we have an autoincrement field.
   So fetch and return the map with this included."
  [m table]
  (sql/with-connection db
   (let [ins (sql/insert-values table (keys m) (vals m))]
     ;; Bind in the returned primary key
     (assoc in-m :id (:generated_key ins)))))

;; Use the function to insert values to the database
(ins {:sample-key "sample value"} :sample-table)

The main thing to notice is that the ins function opens and closes the database connection as a result of living within the with-connection macro. This is important as, not only is it less wasteful or resources, but it means you don't have to worry about managing this resource at a higher/different level of the code. Keeping it simple for us sinners. (Aside: if this opening/closing overhead becomes significant the answer is a thread-pool, but we're not going there today).

The Composability Problem

At first blush it might seem that you could compose this function to affect several database insertions, a la:

(defn double-insert [m1 m2]
  (ins m1 :table)
  (ins m2 :table))

and indeed this will work. However, it opens and closes the database connection for each query, which is time consuming. Worse than this, however, is that this design precludes database transactions on account of each query existing within its own connection! This is a serious disability - we have broken composition.

Connection Wrapping

What we want is for each function that touches the database to check for an existing connection prior to doing anything, using it if its available and otherwise creating a using a new one. Thoughtfully the authors of java.jdbc provide the function find-connection function, probably with exactly this in mind. So we can create the macro.

(defmacro wrap-connection [& body]
  `(if (sql/find-connection)
    ~@body
    (sql/with-connection db ~@body)))

and use it like this

(defn ins
  "Inserts a map into the database. The keys of the map must map onto the fields
of the database.  In most cases we have an
autoincrement field.  So fetch and return the map with this included."
  [m table]
  (wrap-connection
   (let [ins (sql/insert-values table (keys m) (vals m))]
     ;; Bind in the returned primary key
     (assoc in-m :id (:generated_key ins)))))

This is probably the first macro I've ever actually had to create; I usually steer well clear of them. The reason it is necessary is the delayed evaluation of the arguments. If it were a function, wrap-connection, would attempt to evaluate the query, body, when it was called, but the connection wouldn't be set up yet, so it wouldn't work. Macros don't evaluate their arguments upon being called, avoiding this problem. This if we modify our double-insert function to

(defn double-insert [m1 m2]
 (wrap-connection
  (ins m1 :table)
  (ins m2 :table)))

both insertions will happen within a single connection. Note that the first version of double-insert will still work just fine with the new ins, but ins will open and close the db connection, hence you will two connections. This is the point.

Transactions

To convert double-insert into a database transaction with rollbacks etc, we play a similar game.

(defmacro transaction [& body]
  `(if (sql/find-connection)
     (sql/transaction ~@body)
     (sql/with-connection db (sql/transaction ~@body))))

(defn double-insert [m1 m2]
 (transaction
  (ins m1 :table)
  (ins m2 :table)))

Again, this must be a macro to avoid the evaluation of the arguments to transaction.

This is now fully composable. You can have as many nested (transaction (transaction ... (transaction ..))) as you care for and only a single database connection will be used. The outermost transaction will use open/close the database and the inner transactions will use it. Furthermore, if any transaction fails, the outermost will fail and the whole thing will roll-back.

Summary

The initial problem was that creating a database connection created a composition boundary: as soon as any code declares the connection, in so doing it defines the highest level of composition. The solution is simply not to make the connection declaration! We perform this elision by having the code itself figure out the highest level of database interaction, and place the declaration at that level. Pretty cute, no ?

Thanks

Thanks again to @cemerick for reviewing.

Tagged as: Clojure, Composition, Macro, SQL 5 Comments

14Aug/113

Clojurescript AJAX, or is that AJACS ?

Introduction

This short post shows a minimal example of performing asynchronous GETs from Clojurescript, and how to pass pure Clojure data structures over the wire. No XML, no JSON, just Clojure. So its AJAX, with Clojurescript rather than XML; can we call this AJACS As usual the code is all to be had on GitHub.

Structure

For illustration the app performs multiplication. The structure is a Clojurescript front end, with two text inputs for numbers and a shiny green button (stolen shamelessly from Rich's demo) that sends those two numbers asyncronously to a Ring server which multiplies them, conj's the answer into the request map, and returns it for display.

Client Side

The client is contained in ./www/multiplier.html and ./www/src/multiplier.cljs. The same instructions for compling up the Clojurescript apply as for the last post.

The HTML file is pretty dull, not much to note, except that the forms are all done in CSS style, as stolen from Rich (gracias !).

The Clojurescript is more interesting.

Starting at the bottom the main function wires up the Closure event system to the shiny green button.

;; Main entry function
(defn ^:export main []
  ;; Wire up the button
  (ev/listen
   (dom/getElement "multiply-button")
   "click"
   multiply-request))

Its passed the callback multiply-request. That in turn jiggles the DOM to get the values of the two boxes as Numbers. It then places them into the request map.

(defn multiply-request
  "Send a request for multiplication to the server"
  [e]
  (let [x1 (extract-from-dom "multiplier")
        x2 (extract-from-dom "multiplicand")
        request {"x1" x1 "x2" x2}]
    (xhr/send (multiply-api request) multiply-callback)))

The core of the function is the call to (xhr/send ...). This is the main Google Closure asynchronous call. You pass it a URI and a callback, and it GETs the URI and calls the callback with the response. Notice that when we create the URI we simple pass the request map through pr-str, which is the Clojure printer, and pop it on the end of the URL.

defn- form-uri
  "Note nasty side effect code in here."
  [form]
  (let [uri (goog.Uri. "http://localhost:9090")
         qr (. uri (getQueryData))]
    (.add qr "form" (pr-str form))
    (.setPath uri "multiply")))

(defn- multiply-api [form]
  (. (form-uri form) (toString)))

That's the kicker here. I didn't have to turn it into any other intermediate format such as XML or json. The callback is really just the inverse of this process

(defn- extract-response [message]
  (reader/read-string
   (. message/target (getResponseText))))

The inner function gets the string out of the response and passes it the the ClojureScript reader which turns it into a Clojure object. In this case a map.

Server Side

The server is a stripped down Ring server. There's intentionally not much to say here, beyond pointing out the roundtripping code

 (GET "/multiply" {params :params}
       (pr-str (multiply (read-string (:form params)))))

Conclusion

This is pretty cute. The only problem I have is that encoding keywords raw into the URI query is not working on account of the colons. Hence I had to settle for using strings rather than keywords in the request map. I'm hoping somebody will let me know how to fix that. Chas Emerick has pointed out in the comments (before my host lost them) that I should really use a POST rather than a GET here, and I'll look at that in the future.

Tagged as: Clojure, Clojurescript 3 Comments

30Jul/113

Most Basic ClojureScript GUI

Introduction

This quick blog post shows a minimal setup to create a GUI using Clojurescript and the Closure Library GUI functions. In addition shows a working setup wrt directories and the compiler. I'm a web-weenie so this stuff wasn't obvious to me, I hope its helpful. I have put up a git repo for this demo here.

Setup

Directories

The Clojurescript compiler seems to expect files and directories to be in a certain order. This structure works and is sane:

-- root   : put .html file here, compiled .js will go here too
   |
   +- src   : put .cljs here
   |
   +- out   : compiled Closure will go here
   |
   +- css   : put stylesheets here

Compiler

Running the cljsc from the terminal is too slow. My minimalist solution is to run a clojure repl from the clojurescript (not project) directory and recompile manually from there. I have added jline to the java classpath to make this easier. Just grab jline-1.0.jar and put into ./lib, then amend ./script/repl to

 java -server -Xmx2G -Xms2G -Xmn256m -cp 'lib/*:src/clj:src/cljs' jline.ConsoleRunner clojure.main

Line up the appropriate incantation, in my case its

 (require '[cljs.closure :as cljsc])
 (cljsc/build
    "/scratch/clojurescript/hello/src"
    {:output-dir "/scratch/clojurescript/hello/out"
     : output-to  "/scratch/clojurescript/hello/hello.js"})

So each time I update the .cljs file, I go to the terminal, hit up-arrow, it recompiles, go to web browser and refresh. Its not great, and there are better ideas out there, but it works.

Closure Library GUI

Using the Closure Library GUI was not natural for me, being a web-weenie. The most complete documentation is the source code and accompanying demos. Grab it from the subversion repo

svn checkout http://closure-library.googlecode.com/svn/trunk/ closure-library-read-only

You'll find the source code in the

closure-library-read-only/closure/goog/ui

directory. What you really want however are the demos, which are in the

closure-library-read-only/closure/goog/demos/index.html

directory. Point your browser there. The best way I found to get grips with this is to trawl the source for the various demos.

Clojurescript GUI Demo

This is a small barebones demo to show what is necessary to get things hooked up.

Getting this stuff going is a little tricky as you need to identify the correct .css files to import into your .html file and the correct .js files to import into your .cljs. Again, trawl the demos to figure it out.

The HTML file

You'll find hello.html in the root of the repo. This is what the browser loads up. The structure of the file is as follows

Load the appropriate .css files with
```
...
```
tags block. You need common.css for all Closure Library widgets. You don't need demo.css, but I put it in because I like the colours. Then you need a .css for each of the widgets you are using. Its a bit messy figuring out which is which as they don't correspond 1-to-1. Again, look at the demo and copy. I've put the full set of Google provided widget css files into the ./css/ directory for easing this.
Load the appropriate .js files with
```
  goog.require('hello');
```
type tags. (Please excuse the use of scrjpt for script, wordpress wouldn't ignore the tags!) You need base.js from the Google Library. You'll see this is in the ./out directory, and it gets placed there by the Clojurescript compiler. Worry no more about it. Then you need the .js file that Clojurescript will compile from your .cljs file. It will end up in the root directory.
Construct the DOM structure of your page with the
```
...
```
type tags. You'll notice the buttons are done one way and the tab browser another. More on this later.
Finally call the Javascript you have compiled from your Clojurescript

The Clojurescript File

First load up the required .js files. You can figure them out from the associated goog.require() calls in the demos.

(ns hello
  (:require
   [goog.dom :as dom]
   [goog.object :as goog-object]
   [goog.events.EventType :as goog-event-type]
   [goog.ui.ColorButton :as color-button]
   [goog.ui.Tab :as gtab]
   [goog.ui.TabBar :as gtabb]))

Instantiate the Library objects

(def database-button (doto (goog.ui.ColorButton. "Database")
                 (.setTooltip "Database Connection Status")
                 (.setValue "red")))

(def messaging-button (doto (goog.ui.ColorButton. "Messaging")
                        (.setTooltip "Messaging Connection Status")
                        (.setValue "red")))

(def tabbar (goog.ui.TabBar.))

Yeah, looks a lot like Clojure to me too.

Connect these objects to the HTML DOM.
There are two ways of doing this with Closure Library: rendering and decorating. The former creates a DOM in the .js code and then attaches it to the DOM in the HTML at the .render call. The latter creates the DOM in the HTML at loading, and the object attaches to that. The former keeps the presentation and control logic in the javascript side, the latter separates them; choose your poison. I have rendered the buttons and decorated the tab bar to give examples of both. Either way issue
```
 (.decorate tabbar (.getElement goog.dom "top"))
```
This attached the object tabbar, constructed earlier, to the DIV in the html with id "top". Easy peasy.
Wire up the event logic.
```
;;  The database button
(.listen goog.events
           database-button
           goog.ui.Component.EventType/ACTION
           (partial handle-button-push
                    database-connected?))
```
Pretty obvious. Attach an event listener to the ACTION event of the object database button and call the function handle-button-push. That function just toggles some internal state to change the colour of the pushed button. You'll notice that handler code looks rather like Clojure; happy days.

Output

Now compile the .cljs using the incantation given earlier and point your browser at the hello.html page. You'll see this, with luck. Clicking the buttons will toggle the colour between red and green
Clojurescriptdemo

Gotchas

The main block I ran into (and was resolved in -5 seconds by triyo on #clojure, thanks dude) was the interop difference from Clojure. Where in Clojure you write

(.method Object)

this will return the source code of the associated method in Clojurescript. Imagine my surprise. Instead go with

(. Object (method))

You'll see examples of this on lines 32 and 33 of the hello.cljs file.

This is mentioned in the docs.

Tagged as: Clojure 3 Comments

Older Entries »

Edmund Jackson

I'm a coder and entrepreneur based in Cambridge, UK, specialising in statistics and finance.

A Core.logic Primer

Composition of SQL Oueries (and a real use case for a macro)

Introduction

Basic Ideas

The Composability Problem

Connection Wrapping

Transactions

Summary

Thanks

Clojurescript AJAX, or is that AJACS ?

Introduction

Structure

Client Side

Server Side

Conclusion

Most Basic ClojureScript GUI

Introduction

Setup

Directories

Compiler

Closure Library GUI

Clojurescript GUI Demo

The HTML file

The Clojurescript File

Output

Gotchas

Blogroll

Edmund Jackson