I thought about where to take my series about Building a System in Clojure next and realized that I don’t like the format of a blog series all that much. Instead, the format of a book seems like a better choice; one where you, the potential reader, are invited to provide feedback from the very first moment of the writing process. I have already started that process and for now I have transferred the existing articles from the series into the book without much further editing. Over the next couple of weeks, I will be working on making the content more consistent with the book format. The book is available for free on leanpub.com. Iff (if and only if) you find the content to be of value, you can pay a suggested price, but that’s entirely up to you and something you can decide on later.
Last week was great. I had just come back from my trip to the United States the week before, where I attended the Conj and got to hang out with dear friends in both Washington DC and New York City. I used to live in DC in 2009 and 2010, and it was really good to be back. Last Friday, I had a talk at Clojure eXchange in London about my BirdWatch application and my journey to Clojure. I think there should be a recording somewhere, but I haven’t checked it out just yet. The day before that, I had parted from Scala with a farewell letter here on this blog. It took me by surprise how much attention this little letter received 1, considering that it was really only between Scala and me.
Do you remember how we first met, back in 2012? I thought your functional approach was fresh, and different. For a while I believed we were made for each other. A first project was a success; on my own I was comfortable with the good parts of you. But as soon as I started working in teams writing Scala, your immense syntax started drowning me. At first, I took it as a compliment that you tried to please me by offering me to work the way I liked. But then I noticed that it wasn’t something you did for me in particular. Instead, you try to be everybody’s darling by offering every software development paradigm known to man.
TL;DR: I realized how much I disliked the process of logging data structures to the console and then trying to find them and repeatedly commenting out and uncommenting println statements. So I decided to fix that.
Last week, I drew a picture of how I wanted to break apart a monolithic application and instead run different parts of the application in separate processes / separate JVMs. The idea was to have a single client for the connection to the Twitter Streaming API and the persistence of the received Tweets in ElasticSearch, plus multiple machines to serve WebSocket connections to the client. For the communication between the processes, I picked Redis Pub/Sub because its model of communication appears to suit the requirements really well. As cute as the drawing may be, I prefer code (plus drawing), so I took the previous monolith apart over the weekend and put Redis in between for communication. It worked really well and here’s how.
I recently did a lot of transcript proofreading concerning enlightening talk for my talk-transcripts project. The most recent one was Design, Composition and Performance, the tenth transcript of a talk by Rich Hickey. That one in particular got me thinking that there are quite a few ideas I want to adopt in BirdWatch, the application about which I started writing this series of articles. That calls for some redesign, so it’s probably time to make up my mind what exactly it is that I am trying to solve with this application.
As I have no other article to publish this week, I thought a weekly update would be in order. Last week I wrote about making relevant and interesting talks more accessible. In the course of that project, I had eleven talks transcribed so far, four more than when I announced the project last week. Not only have I received great feedback about how appreciated this is, I have also learned a lot myself while proofreading the transcripts.
TL;DR: I wanted to study the content of some talks in written form but there were no transcripts available. However, I believe that the valuable and relevant content of these talks should be accessible to everyone, including those for whom ‘sit back and listen’ might not be the most viable option. To make that a reality, I had transcripts made. So far, I have compiled five talks by Rich Hickey, one talk by David Nolen and another talk by Guy Steele. The transcripts are available on GitHub.
TL;DR: This article covers the usage of Transducers in Clojure, spiced up with some core.async. Here’s an animation that shows the information flow of the composed transducer that we are going to build in this article:
If any of that is of interest to you at all (or if you want to see more animations like the one above), you may want to read the following article.
This weekly update is mostly about organizational issues with regard to publishing articles and getting feedback for unfinished articles.