Self-documenting Makefiles

Make, as arcane as a build tool can be, may still be a good first fit for certain scenarios. “Heresy!”, you say, as you hear a so-called “Bazel expert” utter these words.

The specific problem I’m facing is that I need to glue together the NetBSD build system, a quilt patch set, EndBASIC’s Cargo-based Rust build, and a couple of QEMU invocations to produce a Frankenstein disk image for a Raspberry Pi. And the thing is: Make allows doing this sort of stitching with relative ease. Sure, Make is not the best option because the overall build performance is “meh” and because incremental builds are almost-impossible to get right… but adopting Bazel for this project would be an almost-infinite time sink.

Anyway. When using Make in this manner, you often end up with what’s essentially a “command dispatcher” and, over time, the number of commands grows and it’s hard to make sense of which one to use for what. Sure, you can write a README.md with instructions, but I guarantee you that the text will get out of sync faster than you can read this article. There is a better way, though.

Sample output of the make help command that we will implement in this article.

What if we could provide a make help command that showed an overview of the project’s “build interface”? And what if we could embed such information inside the Makefiles themselves, close to the entities that they document? This idea is neither new nor mine, and it has been written about before by different people. However, I bet that most of you haven’t heard about it before so it’s worth for me to repeat it. And I think that my solution is a bit more comprehensive than others I’ve found. So here you go.

Targets

As I mentioned in the introduction, Make is often used as a command dispatcher: with very little code, you can write what essentially are multiple shell scripts with automatic chaining, all wrapped in one single interface. It’s all pretty terrible, but people are used to this pattern due to Make’s ubiquity and somehow expect it when they face a Make-based project.

To implement this command dispatcher idea, each user-facing action is exposed via a target. These targets tend to be marked as “phony”—i.e. they are targets that produce no outputs of their own. Take a look at this Makefile:

.PHONY: build
build: target/debug/program

target/debug/program: src/main.rs src/lib.rs
	cargo build
        
.PHONY: test
test: build
	cargo test

In the snippet above, the target/debug/program target represents a built file. This target depends on a list of sources and specifies what command to run to generate the output when it is missing or out of date (according to file modification times, yikes). When you type make target/debug/program, you expect the file target/debug/program to exist on disk after the command completes.

But the snippet also shows two phony targets: build and test. When you type make build or make test, you do not expect neither a build nor a test file to be created, no. What you expect is that the project is built and tested. And for this, Make evaluates the dependencies of the phony targets (if any are specified, as is the case for build) and then unconditionally executes any commands in the phony targets (as is the case for test).

With this in mind, the first thing we want to do in our make help command is to document these “special” targets that represent user-facing actions. To do this, we’ll leverage one not-well-known aspect of Make’s syntax: the list of dependencies of a target is cumulative across multiple target definitions of the same name. Basically, these target definitions are equivalent:

# All dependencies in one line.
target/debug/program: src/main.rs src/lib.rs

# Dependencies spread over multiple lines.
target/debug/program: src/main.rs
target/debug/program: src/lib.rs

Knowing this, we can add “extra” lines for a target and use one of those to document the target so that we do not end up with super-long lines. For example, we can do:

target/debug/program: # Builds the program.
target/debug/program: src/main.rs src/lib.rs

And then we are just one grep away from extracting the targets and their documentation:

sed -e's/^\([^: 	]\+\):.*#\(.*\)$/\1 \2/p;d' Makefile | column -t -l 2 | sort

OK fine. It’s a bit more complicated than just grep because we have to reformat the lines a bit and we need to create a nicely formatted table. Also, I know the sed syntax is awful, but I really don’t want to call into Perl or Python as other guides tell you just for this silly string manipulation. There are native Unix tools that can help us here, and they are much lighter-weight.

Variables

All other “self-documenting Makefile” tutorials I found out there focus exclusively on documenting targets. But Makefiles often expose another dimension of their API, and this is the collection of user-settable configuration variables that they accept.

Many Makefiles do things like:

CFLAGS ?= -O2

… to indicate that CFLAGS is set to -O2. But note: the ?= operator invites users to override the variable’s value if they choose to. For example, if the user wanted to build the project in debug mode, they could probably do the following and get the code to build without optimizations and with debug symbols:

$ make CFLAGS="-O0 -g" build

Given that these variables are user-facing, we should document them as well as part of the make help output.

To document variables, we don’t have the luxury of splitting their definition into multiple lines like we did with targets to prevent super-long lines. That said, we can still add comments at the end of the line, like shown below, and those comments won't be part of the variable's default value. It is important, however, to not leave any space between the default value and the comment, or else the spaces become part of the variable's value.

DEVELOPER ?= 0# Set to 1 to enable developer builds.

Like with targets, we are also just one grep away from extracting the variables and their documentation:

sed -e 's/^\([^ 	]\+\)[ 	]*?=[^#]\+#\(.*\)$/\1 \2/p;d' Makefile | column -t -l 2 | sort

Again, more complicated than just a grep, but you get the idea.

Putting it all together

Alright. So now we know how to extract a table documenting targets and a table documenting variables, but these two lists may still be too obscure on their own. Which targets are important? Which variables might the user want to look into first?

To address this deficiency, we can preface those tables with some prose that explains, at a very high level, what to do when interacting with the project for the first time. To implement this, we can write the instructions in a separate file (like a README.md) next to the Makefile, and then have our make help command print out the text file’s contents.

And so without further ado, here is how we can tie everything together:

.PHONY: help
help: # Shows interactive help.
	@cat README.md
	@echo
	@echo "make variables:"
	@echo
	@sed -e 's/^\([^ 	]\+\)[ 	]*?=[^#]\+#\(.*\)$$/\1 \2/p;d' Makefile | column -t -l 2 | sort
	@echo
	@echo "make targets:"
	@echo
	@sed -e's/^\([^: 	]\+\):.*#\(.*\)$$/\1 \2/p;d' Makefile | column -t -l 2 | sort

If you copy/paste this text, beware that there are embedded tabs in it. The ones at the beginning of the line are obvious, but the ones in the [ ] character classes are not. The latter are supposed to be [ <tab>].

Now, have fun with this, but please don’t use Make for new projects if you can avoid it!