Gis A Job! (And I'll give you £500)

Decided that I'd offer a £500 reward to anyone who can provide a useful introduction. Details here: http://johnlawrenceaspden.blogspot.com/2010/10/job-hunting-500-reward.html

---

Anyone in Cambridge need a programmer? Obviously Clojure is a speciality, and my current obsession, but I'm also pretty good with C (especially the embedded variety), microcontrollers, and Python, and I have a particular facility with mathematical concepts and algorithms of all kinds. My obsessions can be pretty quickly changed when I need them to be.

I have a reputation for being able to produce heavily optimised but nevertheless bug-free and readable code, but I also know how to hack together sloppy, bug-ridden prototypes, and I know which style is appropriate when, and how to slide along the continuum between them.

I've worked in telecoms, commercial research, banking, university research, a chip design company, server virtualization, a couple of startups, and occasionally completely alone.

I've worked on many sizes of machine. I've written programs for tiny 8-bit microcontrollers, and once upon a time every IBM machine in one building in Imperial College was running my partial differential equation solvers in parallel in the background.

I'm smart and I get things done. I'm confident enough in my own abilities that if I can't do something I admit it and find someone who can.

I also have various ancient and rusty skills with things like Java, C++, R, Common LISP, Scheme, ML, FORTRAN and Pascal which can be brushed up if necessary. Like all lispers, I occasionally write toy interpreters for made-up languages for fun.

If you're a local company using Java, who might be interested in giving Clojure a try (motivation here, in Paul Graham's classic Beating the Averages), I'd love to try to show you what all the fuss is about.

CV here if you're interested: http://www.aspden.com

I've never used a CV before, having always found work through word of mouth. So I expect that it can be improved. If anyone's got any suggestions as to how it could be better written, do please leave comments or e-mail cv@aspden.com.

EMACS Clojure Colour Scheme

Someone rather kindly said in a private e-mail that they like the syntax highlighting on this site, and requested the relevant .emacs

It's actually a standard colour scheme. Under Ubuntu/Debian you install the emacs-goodies-el package:

$ sudo apt-get install emacs-goodies-el

And then the elisp you need is:

(require 'color-theme)

(color-theme-billw)

I don't know who billw is, but thank you. I like it too. 

To make html files from clojure files, use M-x htmlize-file

Generating XML to make SVG vector graphics files

Clojure makes it easy to generate and parse XML. SVG is a form of XML. And therefore clojure is good for doing vector graphics drawings.

This year's Turner Prize Winner

;; The other day, I wanted to make some graph paper.

;; While experimenting with inkscape, I noticed that svg is actually an xml file format.

;; Since clojure is good with xml, that means that it's actually easier to make such a drawing with a program:

;; Each element of the drawing is represented as a map

(defn make-rect [i j squaresize]
  {:tag :rect
   :attrs {:x (str (* squaresize i))
           :y (str (* squaresize j))
           :width  (str squaresize)
           :height (str squaresize)
           :style "fill:white;stroke:black;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"}})

;; The whole file is represented as a map containing those maps

(defn make-svg [gridsize squaresize]
  {:tag :svg
   :attrs {:width "100%"
           :height "100%"
           :version "1.1"
           :xmlns "http://www.w3.org/2000/svg"}
   :content (for [i (range gridsize) j (range gridsize)] (make-rect i j squaresize))})

;; The library clojure.contrib.lazy-xml will turn the nested map into xml:

(require 'clojure.contrib.lazy-xml)

;; We can use with-out-str to capture the output, which is unaccountably printed
;; rather than given back as a string, and spit to write it to a file.

(spit "squares.svg" (with-out-str (clojure.contrib.lazy-xml/emit (make-svg 10 80))))

;; The nice thing about this is that you can then use inkscape to modify the
;; file, and then diff to work out how to take the modifications back into the
;; program. Does anybody know how to make the emit function format the xml so
;; that the output file is nicely readable?

Latest Collection of Filthy Hacks for REPL Conditioning

I have this set as the maven-clojure-plugin replscript, so it loads before the swank server starts.

I got bored waiting for my REPLs to require everything before I could use them, and so I'm using an agent to require all the namespaces in the background after the swank server is started.

I know that this is a filthy hack, but it hasn't bitten me yet, and Lord, the Roman hyacinths are blooming in bowls and

;; LOAD ME WITH
;; (load-file "/home/john/hobby-code/require-all-snippet.clj")

;; This file conditions a repl in various ways that I do all the time.

;; Firstly we want to 'require' all the namespaces on the classpath
;; This ensures that find-doc and the like will work
(require 'clojure.contrib.find-namespaces)

;; Some namespaces may fail to load, so catch any exceptions thrown
(defn- require-may-fail [ns]
  (try
   (print "Attempting to require " ns ": ")
   (require ns)
   (println "success")
   (catch Throwable e (println "couldn't require " ns "\nException\n" e "\n\n"))))

;; Generally we'd want clojure.*, clojure.contrib.*, and any project-specific namespaces
(defn require-all-namespaces-starting-with [strng]
  (doall (map require-may-fail 
              (filter #(. (str %) startsWith strng) 
                      (clojure.contrib.find-namespaces/find-namespaces-on-classpath)))))

;; Some of clojure's extra namespaces are so useful at the REPL that I want them
;; to have shorter names, i.e. I want to be able to type 'r/source' rather than
;; 'clojure.repl/source'.
;; This also means that emacs tab completion can find them with e.g. r/<TAB>

(require '(clojure [test :as t]
                   [inspector :as i]
                   [repl :as r]
                   [pprint :as pp])) 
(require '(clojure.contrib
           [trace :as cct]
           [repl-utils :as ccr]))


;; It drives me up the wall that it's (doc re-pattern) but (find-doc "re-pattern").
;; Can use macros so that (fd re-pattern) (fd "re-pattern") and (fd 're-pattern) all mean the same thing
(defn stringify [x]
  (println "stringify given" (str x))
  (let [s  (cond (string? x) x
                 (symbol? x) (str x)
                 (and (list? x) (= (first x) 'quote)) (str (second x))
                 :else (str x)) ]
    (println (str "translating to: \"" s "\""))
    s))


;; Sometimes I like to ask which public functions a namespace provides.
(defn ns-publics-list [ns] (#(list (ns-name %) (map first (ns-publics %))) ns))
;; And occasionally which functions it pulls in (with refer or use)
(defn ns-refers-list  [ns] (#(list (ns-name %) (map first (ns-refers %))) ns))

;; Nice pretty-printed versions of these functions, accepting strings, symbols or quoted symbol
(defmacro list-publics     
  ([]   `(clojure.pprint/pprint (ns-publics-list *ns*)))
  ([symbol-or-string] `(clojure.pprint/pprint (ns-publics-list (find-ns (symbol (stringify '~symbol-or-string)))))))

(defmacro list-refers
  ([]   `(clojure.pprint/pprint (ns-refers-list *ns*)))
  ([symbol-or-string] `(clojure.pprint/pprint (ns-refers-list (find-ns (symbol (stringify '~symbol-or-string)))))))

;; List all the namespaces
(defn list-all-ns [] (clojure.pprint/pprint (sort (map ns-name (all-ns)))))
;; List all public functions in all namespaces!
(defn list-publics-all-ns [] (clojure.pprint/pprint (map #(list (ns-name %) (map first (ns-publics %))) (all-ns))))

;; With all the namespaces loaded, find-doc can be overwhelming.
;; This is like find-doc, but just gives the associated names.
(defn find-doc-names
  "Prints the name of any var whose documentation or name contains a match for re-string-or-pattern"
  [re-string-or-pattern]
    (let [re  (re-pattern re-string-or-pattern)]
      (doseq [ns (all-ns)
              v (sort-by (comp :name meta) (vals (ns-interns ns)))
              :when (and (:doc (meta v))
                         (or (re-find (re-matcher re (:doc (meta v))))
                             (re-find (re-matcher re (str (:name (meta v)))))))]
               (print v "\n"))))




;;find symbol or string in docs 
(defmacro fd [symbol-or-string] `(find-doc (stringify '~symbol-or-string)))

(defmacro fdn [symbol-or-string] `(find-doc-names (stringify '~symbol-or-string)))

;; find the source file which defines a thing:
(defn source-file* [symbol] (:file (meta (resolve symbol))))

(defmacro source-file [symbol-or-string] `(source-file* (symbol (stringify '~symbol-or-string))))

;;get the methods of a java object
(defn meths [x] (println (apply str (interpose "\n" (map str (.getMethods (if (class? x) x (class x))))))))
;;get just the names of the methods
(defn meth-names[x] (map #(.getName %) (.getMethods (if (class? x) x (class x)))))


;;debugging macro                                try: (* 2 (dbg (* 3 4)))
(defmacro dbg [x] `(let [x# ~x] (do (println '~x "->" x#) x#))) 
;;and pretty-printing version 
(defmacro ppdbg [x]`(let [x# ~x]
                      (do (println "--")
                          (clojure.pprint/pprint '~x)
                          (println "->")
                          (clojure.pprint/pprint x#)
                          (println "--") x#))) 

;;and one for running tests 
(defmacro run-test [fn] `(test (resolve '~fn)))

;; def-let as in blogpost
(defmacro def-let
  "like let, but binds the expressions globally."
  [bindings & more]
  (let [let-expr (macroexpand `(let ~bindings))
        names-values (partition 2 (second let-expr))
        defs   (map #(cons 'def %) names-values)]
    (concat (list 'do) defs more)))


;; Sometimes it's nice to check the classpath
(defn- get-classpath []
   (sort (map (memfn getPath) 
              (seq (.getURLs (java.lang.ClassLoader/getSystemClassLoader))))))

(defn print-classpath []
  (clojure.pprint/pprint (get-classpath)))

(defn get-current-directory []
  (. (java.io.File. ".") getCanonicalPath))

;;print the classpath
(println "Classpath:")
(print-classpath)

(println "Current Directory" (get-current-directory))

;;print the public functions in the current namespace
(println "Current Namespace")
(list-publics)

;;hint on how to require project specific namespaces
(println "to require all namespaces starting with example:")
(println "(require-all-namespaces-starting-with \"example\")")

;; see http://blog.n01se.net/?p=85
(println "setting *print-length* to 103, *print-level* to 13 to deal with infinities")
(println "you have to do this before starting the swank server if you want it to work in emacs evaluations")
(set! *print-length* 103)
(set! *print-level* 13)

;; but we don't need this bit, 
;; (require 'clojure.contrib.repl-utils)
;; (clojure.contrib.repl-utils/add-break-thread!)
;; because swank repl threads already have a break handler set.
;; might come in useful for command line repls though.


;;require everything from clojure and clojure.contrib, so that find-doc can find it. Do it in an agent so it doesn't block repl startup. Jesus, am I really allowed to do this?
(def require-all-agent (agent "not done"))
(send-off require-all-agent (fn[agent] (with-out-str (require-all-namespaces-starting-with "clojure"))))

Cambridge Clojure Meetup (Tuesday 5th October in the Punter)

We thought we'd try and revive the defunct Cambridge Clojure meetings.


Who fancies a pint on Tuesday evening at 19:30 in the Punter?






View Larger Map

Polyglot Maven with Clojure and Emacs : My Edge is Bleeding

After a couple of hours hacking, I admit defeat. Google is not my friend any more. I am inches from my goal, and yet so far ....

I decided to try out polyglot maven, which promises the joy of maven without the xml of maven.

It's still pre-release, so it can't be criticized for the fact that its instructions don't work, or for the fact that build from source is broken.

However, it is almost possible to figure out what to do:

Download the prebuild 0.8:
$ wget http://polyglot.sonatype.org/site-nexus/download/pmaven-0.8-SNAPSHOT-bin.zip


Unzip it:

$ unzip pmaven-0.8-SNAPSHOT-bin.zip

Run it, using the pom.clj below as the project file, and ask it to open a swank server:

$ pmaven-0.8-SNAPSHOT/bin/mvn -f pom.clj clojure:swank

Connection opened on local port  4005

#<ServerSocket ServerSocket[addr=localhost/127.0.0.1,port=0,localport=4005]>

All is well. This works on my machine, running Ubuntu 10.04 in a guest account.

But I have cheated. Crucial jars live in the clojars repository.
I cannot figure out how to add the clojars repository to my pom.clj.

However, if you run maven on a corresponding pom.xml, with the same version and that repository listed, then it will download the relevant jars and put them in the local repository, which is good enough. After that maven will find them.

Polyglot maven understands the pom.xml file below as well. Just put it in your current directory and run:

$ pmaven-0.8-SNAPSHOT/bin/mvn -f pom.xml clojure:swank

And that will bring the necessary jars into your local repository. After that the pom.clj file is enough.

I am not claiming that this is in any way an improvement yet, but it looks very promising.

If someone can tell me what to add to pom.clj to add clojars, we are done.

I have grown rather fond of maven's pom.xml files. Soon they will be gone. I will actually miss them.

But brevity is the soul of not cocking things up. Here are the two files:

pom.clj

(defproject main "com.aspden:polyglot-maven-test:1.0-SNAPSHOT"
    :dependencies [["org.clojure:clojure:1.3.0-alpha1"]
                   ["org.clojure:clojure-contrib:1.2.0-SNAPSHOT"]
                   ["swank-clojure:swank-clojure:1.3.0-SNAPSHOT"]
                   ["jline:jline:0.9.94"]]
    :plugins [["com.theoryinpractise:clojure-maven-plugin:1.3.4"]])
 

pom.xml

<project>

  <modelVersion>4.0.0</modelVersion>
  <groupId>com.aspden</groupId>
  <artifactId>polyglot-maven-test</artifactId>
  <version>1.0-SNAPSHOT</version>
  <name>polyglot-maven-test</name>
  <description>polyglot-maven-test</description>


  <repositories>
    <repository>
      <id>central</id>
      <url>http://repo1.maven.org/maven2</url>
    </repository>
    <repository>
      <id>clojure</id>
      <url>http://build.clojure.org/releases</url>
    </repository>
    <repository>
      <id>clojars</id>
      <url>http://clojars.org/repo/</url>
    </repository>
  </repositories>

  <dependencies>
    <dependency>
      <groupId>org.clojure</groupId>
      <artifactId>clojure</artifactId>
      <version>1.3.0-alpha1</version>
    </dependency>
    <dependency>
      <groupId>org.clojure</groupId>
      <artifactId>clojure-contrib</artifactId>
      <version>1.2.0-SNAPSHOT</version>
    </dependency>
    <dependency>
      <groupId>jline</groupId>
      <artifactId>jline</artifactId>
      <version>0.9.94</version>
    </dependency>
    <dependency>
      <groupId>swank-clojure</groupId>
      <artifactId>swank-clojure</artifactId>
      <version>1.3.0-SNAPSHOT</version>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>com.theoryinpractise</groupId>
        <artifactId>clojure-maven-plugin</artifactId>
        <version>1.3.4</version>
      </plugin>
    </plugins>
  </build>
  
</project>

Clojure Leiningen Setup for either EMACS or REPL

Installing Leiningen

Install maven:

$ sudo apt-get install mvn2

Download the leiningen master script:

$ wget http://github.com/technomancy/leiningen/raw/stable/bin/lein

Make it executable:

$ chmod +x lein

Run it:

$ ./lein

There will be a certain amount of churning as leiningen downloads stuff.

That's the installation part over with.

Trying it out

Now create a file project.clj, where you specify which versions of everything that you want.

(defproject trial "1.0.0-SNAPSHOT"
  :description "FIXME: write"
  :dependencies [[org.clojure/clojure "1.2.0"]
                 [org.clojure/clojure-contrib "1.2.0"]]
  :dev-dependencies [[swank-clojure "1.2.0"]])

Make sure you don't have a subdirectory libs, or leiningen will destroy it without warning.

If you don't, then type:

$ ./lein deps

You will see the unmistakeable signs of maven downloading the internet. Make tea.

Once all that's done, you can start a command line REPL with:

$ ./lein repl

"REPL started; server listening on localhost:48202."
user=> (clojure-version)
"1.2.0"
user=>

We have a working clojure, with contrib libraries on the classpath.

Anyway, using the REPL directly is not a great deal of fun, so kill it with Ctrl-D, and now we try

$ ./lein swank
user=> Connection opened on local port  4005
#<ServerSocket ServerSocket[addr=localhost/127.0.0.1,port=0,localport=4005]>

And we can connect directly to this from emacs with M-x slime-connect

If you need help setting up emacs to use slime and clojure-mode, which is very easy these days, then see the second half of my earlier post which covers how to set up emacs and clojure with maven. The emacs instructions are exactly the same, and I've just re-checked them to make sure they still work (it only takes 2 minutes).

---

Caveats

I am a new user of Leiningen, having been an entirely happy maven user for the last year.

I am tempted by the short project files and by the fact that leiningen appears to be the most common way to run clojure amongst clojure fans, but discomforted by the idea of putting an abstraction layer over the beautiful simplicity of maven. That sort of thing tends to set my hair on fire.

On the other hand, who better than a sceptical newbie to write an installation tutorial?



Notes:

I've tried this in a clean guest account under Ubuntu 10.04. It probably works the same way on other UNIXes, and I imagine that something similar might work on Windows.

Don't call your new project "test"! I did at first, and that resulted in some weird errors because the name was conflicting with a clojure built-in.
So I called it trial instead.

Weirdly, the REPL that is produced here sometimes forgets to give me a prompt, leading me to believe that it's hung. Typing something (particularly close brackets, in case it's waiting for you to finish something you started typing earlier) tends to wake it up.

You can put the lein script somewhere on your path instead of running it directly with ./lein . I have a ~/bin directory where executables live, and I'll put it there.

If you change project.clj, leiningen won't notice until you redo ./lein deps. At least I think that's what's happening.

After running the above, you can look around to see what it's done.
As far as I can tell:
It has created a local maven repository in ~/.m2, where jar files are all stored in the usual directory structure organised by their version number.
It has made physical copies of some of these files (not soft links as you'd expect) in the libs subdirectory that it has created, and these are the ones that it puts on the classpath for running clojure. I think this is what the lein deps step is about. I'm not quite sure why it does this. It seems ugly, but maybe it safeguards the copies in the maven repository against accidental changes?
It's made a .lein directory, in which there are no files.


I don't think the ./lein deps step is necessary if you type ./lein repl. It seems to do it for you.

On the other hand, if you change the project.clj file, then ./lein repl doesn't react to the changes. You need to do ./lein deps manually.

As soon as I create project.clj, I immediately want to type ./lein swank, to start a clojure image that can talk to emacs, but for some reason this doesn't work. Leiningen says:
That's not a task. Use "lein help" to list all tasks.
You have to do either ./lein repl or ./lein deps first.

The main attraction of leiningen for me at the moment is the short project.clj files compared to maven's verbose pom.xml.

There is a polyglot Maven project, aiming at exactly this problem, which I shall have to investigate. But it is almost certainly worth learning to use leiningen, which is so widely popular amongst Clojure developers. It may have advantages due to its clojure-specifity. Does anyone know of any interesting things that it can do that maven-clojure-plugin either can't or makes difficult?

I have heard recently of other clojure-specific dependency management tools, like cake and cljr. I don't know anything about them, but they sound interesting. Does anyone have any opinions about them to share?