Introducing Avout: Distributed State in Clojure

Posted by on Nov 29, 2011

Today we are releasing Avout, which brings Clojure's in-memory model of state to distributed application development by providing a distributed implementation of Clojure's Multiversion Concurrency Control (MVCC) STM along with distributable, durable, and extendable versions of Clojure's Atom and Ref concurrency primitives.

Avout enables techniques that require synchronous, coordinated (i.e. transactional) management of distributed state (see also JavaSpaces, complementing approaches that focus on asynchronous, uncoordinated communication between distributed components, e.g. message queues (0MQ, RabbitMQ, HornetQ), event-driven approaches (Netty, Aleph), and actors (Erlang, Akka.

Much has been written [1, 2, 3] on functional programming and the advantages of designing programs that emphasize pure functions and immutable values, and that minimize or eliminate, wherever possible, mutable state. Of course, it's not always possible to completely eliminate the need for mutable state, and that's where Clojure's precise model of time, identity, and state becomes powerful.

Likewise, when designing distributed applications, it is desirable to create components that are loosely coupled and that communicate with each other asynchronously, but this too is also not always possible. There are times when you need coordinated access to state across systems in a distributed application, and this is where Avout comes in.

Avout grew from work on zookeeper-clj, a Clojure DSL for working with Apache ZooKeeper. The functionality provided by ZooKeeper has proven important when building distributed applications at Relevance, so zookeeper-clj was developed to ease future development with it. And with the DSL developed, it was natural to implement several common ZooKeeper recipes, including leader-election and distributed locks, as a standalone Clojure library.

It was during a Clojure/core Friday stand-up meeting, when I was describing the work on distributed locks, that Rich Hickey strongly suggested not stopping there, but continuing forward by implementing distributed versions of Clojure's Atom and Ref concurrency primitives, which of course would mean implementing a distributed version of Clojure's MVCC STM. How hard could that be?! :)

Stuart Sierra and I had discussed this idea a year earlier, but we knew that we would need an implementation of distributed locks just to get started, and so never moved forward with the idea. But now I had distributed locks, some time to dedicate to the task, thanks to Relevance and Clojure/core, and Rich's encouragement to pursue the idea, so I did.

Several weeks later, after digging into the design of Clojure's STM, consulting with Rich, gathering feedback from others at Relevance and at the Conj, and performing a lot of testing and tuning, Avout was born.

Using Avout

Below is the Avout equivalent of Hello World.

    (use 'avout.core)
    (def client (connect ""))

    (def r0 (zk-ref client "/r0" 0))
    (def r1 (zk-ref client "/r1" []))

    (dosync!! client
      (alter!! r0 inc)
      (alter!! r1 conj @r0))

Start by creating a ZooKeeper client with the connect function, then create two ZooKeeper-backed distributed Refs using the zk-ref function. Finally, perform a dosync!! transaction that updates both Refs with alter!!. Using Avout isn't much different than using Clojure's in-memory Atoms and Refs.

Avout Atoms and Refs implement Clojure's IRef interface, and therefore support functions that operate on IRefs, including: deref (and its reader-macro, @), set-validator!, add-watch, and remove-watch.

Avout also provides "double-bang" versions of the remaining core Atom and Ref functions (reset!, swap!, dosync, ref-set, alter, commute) for use with distributed Atoms and Refs, reset!!, swap!!, dosync!!, ref-set!!, alter!!, commute!!.

Note: Avout Refs cannot participate in in-memory dosync transactions, but Avout's local-ref provides the equivalent of an in-memory Ref that can participate in dosync!! transactions with distributed Refs.

Extending Avout

Two types of Atoms, zk-atom and mongo-atom, and three types of Refs, zk-ref, mongo-ref, and local-ref have been implemented, and Avout can be extended with additional types of Atoms and Refs that use different containers for their state, durable or not, including (No)SQL databases, (distributed) filesystems, in-memory data structures, and RESTful webservices. The types of values supported by each depends on both the backend store and the method of serialization used, and transactions containing different types of Avout Refs are supported.

New types of Atoms can be created by implementing the avout.state.StateContainer protocol,

    (defprotocol StateContainer
      (initStateContainer [this])
      (destroyStateContainer [this])
      (getState [this])
      (setState [this value]))

and new Ref types can be created by implementing avout.state.VersionedStateContainer.

    (defprotocol VersionedStateContainer
      (initVersionedStateContainer [this])
      (destroyVersionedStateContainer [this])
      (getStateAt [this version])
      (setStateAt [this value version])
      (deleteStateAt [this version]))

More on Avout

To learn more about Avout and distributed-state in Clojure, visit


  1. Provide Avout Java API
  2. Optimize commute!! implementation, currently commute!! just calls alter!!
  3. Implement distributed agents
  4. Experiment with other back-end state-stores, e.g. Terracotta, JavaSpaces
  5. Abstract out ZooKeeper functionality and implement a version of Avout without dependencies or one with other dependencies, e.g. JGroups


  1. Out of the Tar Pit
  2. Clojure State
  3. Okasaki Publications

blog comments powered by Disqus