The Lost Chapter: A Macro System For Monkey
If you don’t care about the Who, Where, When, Why, How and the Why Is It A Lost Chapter? and want to skip to the What: I wrote a new chapter for Writing An Interpreter In Go and you can read it for free at interpreterbook.com/lost. Otherwise, read on…
The pages you are about to read were found amidst the rubble of a collapsed ruin. Wedged between the scratched and battered cases of old machines once called “computers”. Bearing, in faint white and barely readable, the title “Writing An Interpreter In Go. Chapter 5: A Macro System For Monkey.” …
Alright, I’ll admit it: that was a lie. What I want to show you is not really a lost chapter, preserved through the eons, found in the ruins of a long-gone civilization. I just needed a good intro.
You see, I couldn’t sit still. In the first couple of months after publishing Writing An Interpreter In Go I took some time off from Monkey, the programming language we built in the book. “The book is done. Take a breath and play around with something else. After working on it for a year you deserve it”, I told myself, only to grow more anxious by the week about all the features, optimizations and tweaks I could try and add to Monkey. In the end, the temptation of everything Monkey could still be won. I gave in and restarted work on Monkey again.
This resulted in two things: a project I’m not ready to talk about yet and a new, additional chapter for Writing An Interpreter In Go called The Lost Chapter: A Macro System For Monkey, which I want to tell you all about.
It started with me getting sidetracked while working on said secret project by discovering how elegant and beautiful macros in Racket are. I guess, I just can’t stop myself from ushering an impressed “nice” when hearing about “code that writes code”. Next thing I know I was digging through various implementations of macros in different languages and getting more and more fascinated. It’s code that writes code! It’s a hand that draws itself! How could I not be fascinated by that?
A few “Huh, interesting…” followed by more “Well, I guess, it wouldn’t be too hard to just…” later I successfully added macros to Monkey. Macros that are able to modify and generate Monkey source code and are evaluated in their own macro expansion phase. A real, Lisp-style macro system. I was elated.
In fact, the whole journey from learning about how macros are implemented and why they’re so powerful to implementing them myself was so mind-blowing and fun that I had to write about it.
At first I thought I was writing a blog post or a tiny addition to the book and gave it the working title “The Lost Appendix”, thinking of a few pages hidden at the end of a book.
It ended up with the title The Lost Chapter: A Macro System For Monkey, because what we have here is not a small addition. It’s a complete chapter, close to 50 pages in PDF format, that shows you how to implement a fully-working macro system for Monkey - step by step, all code shown, fully tested, just like the book. You can think of it as the fifth chapter of Writing An Interpreter In Go, since it seamlessly continues the previous four. It’s just being delivered a few months later than the rest of the book.
But why “The Lost Chapter”? Because a text about macros deserves a touch of mystery, don’t you think? It’s code that writes code, come on! It’s snakes eating their own tail and surgeons operating on themselves! If that isn’t worthy of title that’s a little bit out there, I don’t know what is.
I also didn’t want to make it an addition to the book itself. On the practical side there’s the hurdle of extending a paperback edition by around 50 pages and not being able to send the update to readers who already bought the paperback. But then there were also, let’s say, “conceptual” considerations.
While I consider learning to build your own programming language a worthwhile endeavor that can teach you a lot of valuable things about programming, I’ll concede that it looks pretty disconnected from the realities of one’s day job. But adding a macro system? Writing code that lets you write code that writes code? That doesn’t just look unrealistic, but rather … Let me put it this way: totally and completely nuts and, oh, incredible fun!
I wanted this chapter to be exactly that: a fun addition to Writing An Interpreter In Go, not quite Monkey canon, but a bizarro expansion pack; a curious and accidental supernova in the same universe.
Oh, and did I mention it’s available for free? Well, it’s available for free. Read it online or download it as PDF/HTML/Mobi/ePub here:
The downloadable version also includes all the runnable, tested code shown in the chapter and the complete Monkey interpreter from Writing An Interpreter In Go.
I hope it’ll get you to usher a “nice”, too.