Literate Dotfiles

October 4, 2017

Software
DevOps

In short: I keep my personal computer setup files, called dotfiles, in one document that mixes plain-English notes with the actual code. A tool named Babel then pulls the code out to build the real files on my machine. I have used this for a few months and find my setup much easier to read, understand, and change with confidence.

Motivation

Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.

In Donald Knuth’s 1984 paper “Literate Programming”, he argued that code is not enough. That is to say it is not enough to only tell a computational device how to behave. The cognitive energy of the humans who maintain such code also need to be considered. Such was the conundrum I found myself in while trying to maintain my ‘dotfiles’ - my user-specific system configuration files for *nix-like hosts. I’ve struggled to find a balance of organization, readability, and documentation.

Early on, I treated these configuration files as write-only repositories of code snippets. As a result, they were difficult to reason about so I didn’t try to change them too often. They were ‘code’ in the deepest sense of the word - I never intended them to read well. As such, the few times a friend or colleague would ask about a particular piece of behavior while pairing, it was almost impossible to discuss with colleagues - the configuration itself was a lousy example.

A step toward literacy

What if, with a little reorganization, we could document intent, in a human readable format, and provide executable examples from which our actually-on-disk dotfiles could be generated?

Enter Emacs’ Babel package; a sub-system of Emacs org-mode which allows us to embed code blocks inside of natural language documents. Exporting the code is great, but what about the reader’s experience? Naturally, emacs handles the display of org-mode documents rather well. When working with the dotfiles org-mode document, one has access to all of the normal Emacs and org-mode document editing features: org’s easy to use document syntax, section/hader folding, search, etc. But what about our non-emacs friends? Popular git forges like Github and Gitlab have us covered. In addition to markdown, org-mode tends to be supported as a source format for documentation and static HTML generation on popular code hosting platforms. As a result, the woven/human-readable version of the dotfiles render rather well.

Making the move

Organizing a scattered collection of dotfiles configuration into a literate document and generating/tangling the code examples into usable configuration can be described as follows:

First, organize and copy each logical group of configuration code the existing dotfiles into org-mode code blocks.
Then add intention revealing documentation for each block.
Next, add metadata about the tangling/export process. These properties tell babel that the code blocks under the .bashrc header should be exported to the file at ~/.bashrc.
And finally we need to tangle/export the code by running M-x org-babel-tangle - in my dotfiles repository I include a dotfiles generation and installation script that will automatically tangle/export all of the code blocks.

Conclusion

I’ve used this literate approach to generate my personal dot-files and Emacs configuration for a few months. I find it to be much easier to manage and reason about and maintain confidently. Though I wouldn’t recommend Literate Programming for every task, I find that “prose first development” works great for clarifying the scope and intent of a task when modifying system configuration.

UPDATES:

<2021-07-11 Sun> Update links to Emacs configuration. Since the original posting of this article, I’ve consolidated my .emacs.d configuration into my .files repository - IE they’re all one repository. This change updates the hyperlinks to make sure the reader lands in the correct location.
<2026-07-04 Sat> I no longer use the literate approach described here. Since this writing I’ve refactored both my .files and Emacs configuration into hand-written, modular Emacs Lisp files rather than tangling them from a literate org document. The “Emacs configuration” link above is pinned to a commit from the original literate version, so it still reflects the approach this article describes; the .files repository itself now reflects the newer modular layout.

Glossary

literate programming: writing code and a human-readable explanation of it together in a single document.
*nix: Unix and Unix-like operating systems (for example Linux and macOS).
org-mode: an Emacs document format for notes and writing that can also hold and run embedded code.
Babel: the Emacs org-mode feature that runs embedded code blocks and extracts them into real files.
tangling: extracting the code blocks from a literate document into standalone source files.
woven: the readable, formatted document produced from a literate source, as opposed to the extracted code.
git forge: a website that hosts version-controlled code repositories, such as GitHub or GitLab.