Why I chose Common Lisp over Python, Ruby, and Clojure
A few months ago, two co-founders (Stu Wall and John Buchanan) and I (Shaneal Manek) started working on a startup called Postabon.
The idea behind Postabon is simple: we wanted to create a platform where users could find and share ‘deals’ at brick and mortar stores (be they sales, coupons, happy hours, specials, etc). For example, if I’m out near a mall and need a pair of jeans I can pull out my phone and see which store near me is having the best sale on pants right now.
I just wanted to talk about a few of the high level technical decisions that I’ve made – in the hopes that it could help other people starting new projects out (and that I can get some feedback and learn something myself). This post is going to be pretty tightly focused on the language I chose. I have a few other posts in mind on topics such as the database (BerkelyDB) and overall architecture that I’m planning to write up in the next week or two..
Language
The way I saw it, was that as the sole programmer I needed a language that was concise, powerful, and that let me work quickly. This set of requirements, in my mind, eliminated Java (and I don’t know any C# …), which left me considering Python, Ruby, Clojure, and Common Lisp. I thought Haskell and Erlang were promising – but I’m just too inexperienced with them to commit to a large project (and, for better or worse, they aren’t really known as great languages for web applications).
Python
I am fairly experienced with Python, there are lots of great libraries/frameworks for anything I would want to do, and it would be easy to bring other programmers on-board later. However there were a few negatives that, in aggregate, were enough to get me to move on.
First, and most importantly, the Python 2 to 3 conversion really scared me. Most libraries I wanted to use were still Python 2 only – which meant I would have had to write Postabon’s back end in Python 2. But it makes no sense to me to write a large app, that I may have to maintain for years, in a language that has effectively received a death sentence. Python 2 is fine now – but in the coming months and years new libraries, features, and performance improvements are only going to be introduced in Python 3, and I didn’t want to get left behind or forced to take on an expensive and time consuming port in the future.
Second, I know it’s a bit cliche, but I don’t like the Global Interpreter Lock, which makes it basically impossible to write multi-threaded apps that work on multiple CPUs. Of course, writing a multi-process app would be a reasonable work-around, but it is a bit of an annoyance.
Finally, Guido’s disdain for functional programming makes it clear that I would be a second class citizen in Python-land. As a few small examples see:
- The fact that he proposed removing map, reduce, filter, and lambda from Python 3.
- His refusal to include tail call optimizations in the language, despite the minimal down side.
- The completely broken implementation of closures.
My mind just works functionally, and I don’t want to be forced to fight the language I’m using at every turn.
Ruby
I’ve played with Ruby (mostly in the context of Rails) some – although I’m nowhere near as proficient with it as I am with Python, Lisp, etc. Ruby has a lot of the same strengths as Python, with fewer weaknesses. My criticisms about Python’s GIL apply to it too – but again simply using processes is an acceptable work-around.
The biggest reason I chose not to go this route is that the Ruby community is just moving too fast for me right now. Some major component of the development/production stack of choice seems to be changing every 6 months (e.g., I’ve seen the webserver go from FastCGI/Apache to Mongrel to Phusion to Unicorn). I couldn’t even easily figure out which version (1.8 or 1.9?) to use – or even which implementation (Ruby MRI, Ruby EE, JRuby, etc). Most of the articles I found online are a few months old and I am told they are no longer accurate.
Also, much of the Ruby community is built around Rails, and I’m a bit wary of using ‘heavy weight’ frameworks like Rails (or Django) on large custom projects. In my experience they make the first 90% of what I’m trying to do be really easy – but then make the last 10% a living hell since I need to modify something the framework never intended me to control. I probably could have written a ‘bare-bones’ implementation of the site’s back-end in Rails in a week instead of two weeks, but I would rather ‘waste’ that one week up front to have more flexibility later.
For example, I ended up writing my own completely stateless session handling, building a fairly smart geo-spatial cache (in-memory R* trees that asynchronously persist to disk using B-Trees), and using a key-value store and raw b-trees for persistence (instead of a relational database). These (and a lot of other non-standard decisions I’ve made) are possible within Rails, but I think they would have cost me more time and energy than Rails would have saved up front – especially in light of my lack of experience with Rails.
In principle, I could see Ruby being the right choice for someone who was more experience with it upfront, is adequately plugged into the community and willing and able to switch out components of their stack. But, personally, I prefer a bit more stability in things.
Clojure
There are a lot of great things about Clojure: it runs on the JVM so I get all the Java libraries and that great JVM performance, it’s functional from the ground up, and it even has macros.
However, 6 months ago Clojure hadn’t even had it’s 1.0 release (and the language was constantly changing). When I tried to download it the Slime integration was completely broken and I had to manually search through the SVN repos of several key components to find a relatively recent working set of tools that worked together.
My feeling is that things are better now (Clojure is 2 years old!) and if I were making this decision again today, I would give much more serious consideration to Clojure.
Common Lisp
Finally, that brings me to Common Lisp. I have plenty of experience writing web apps in Lisp, so the high barrier to entry wasn’t a deterrent in my case. Although, make no mistake, that learning curve for writing a good CL web app is steep enough that I would warn most programmers to shy away from Lisp for writing a production app on a tight schedule (I would like to write a post or two that help new users get over the hump though).
I like that the language stabilized 15 years ago, that the kinks have been worked out, and that it has stood the test of time. I know the traditional complaint is the dearth of libraries, and there obviously aren’t as many options as they may be for other languages. While I have been fortunate to find several great options for everything I’ve needed to do so far (JSON/XML parsing, HTTP servers and clients, ORMs, etc) – more obscure libraries like Thrift and OpenID support may be an issue in the future. The lack of libraries is, without a doubt, the biggest disadvantage of CL and one of the reasons Clojure is so appealing to me. I can usually just write my own foreign function interface into a C library – but that’s really time consuming compared to downloading an egg/gem/jar.
The ability to use dynamic typing for most of my code but optionally give the compiler type-hints and get all the performance of a statically typed language for critical portions is a killer features that I still haven’t found elsewhere.
On balance, I think that Common Lisp was the best choice for me given my background and the needs of this project.
Status
Postabon just publicly released in New York! We are up to about 5K lines of Common Lisp (and well over 15K LoC total, including a website, mobile website, and iPhone App) and things have been progressing smoothly.
The next post I’ll write (probably in a few days) would cover our site’s general architecture. After that, I was planning to dig into some specifics about the best way to manage/deploy a production Common Lisp webapp, which I hope would help a new Lisper get off the ground.
