Monorepos: Version, Tag, and Release Strategy

If additional CI work is the #1 (initial) side-effect of moving to a monorepo, then dealing with versioning, tagging, and releasing is a close #2.

Talking about versioning or tagging strategies is not exciting stuff but seeing as we just dealt with it and it’s fresh in my head, I might as well document the solution and help someone retain some sanity as they navigate versioning, tagging and release strategies for their monorepo.

Some Context

In late December 2023, we migrated ~10 public repositories to a single monorepo. This effort was documented in these two posts:

All of the expected stuff happened — CI, docs, scripts, Makefiles, and whatnot had to be updated to work with the new monorepo directory structure. However, one thing we did not anticipate is the amount of time we’ll need to spend on figuring out and fixing our version, tag, and release process.

This was our list of requirements:

Each individual component needs to have its own version.
Each individual component can have its own release process.
It should be possible to have a version for the “whole” monorepo. – As in, a single version that can be used as a point-in-time reference for the state of the entire repository.

This list of demands is fairly normal but as it turns out, it can be a serious pain in the ass. So here follows a strategy that did work for us + the problems you’re likely to hit + workarounds for those problems.

⚠️ The version/tag/release headaches can be avoided if you can use a single, unified version. ⚠️

But… we decided against this for a few reasons:

We would have to “build the world” every time there is an update to the repo. This is not feasible as some of the components take >15min to build.
We did not want to have a “tag explosion” — the repos are updated fairly often and we would end up with thousands of tags in a very short period.
It would be difficult to “spot” what is considered a “good” tag that represents a fully working project.

Unified versions seem simple at first, however they come with a bunch of caveats and gotchas.

Our Strategy

This is the versioning, tagging, and release strategy we settled on:

Sub-component CI PR workflow is gated for: – Creation of a pull request AND – Detecting changes (path filter) in apps/$project/**
Sub-component CI release workflow is gated for: – Push to main (there is no “merge” trigger) AND – Detecting changes (path filter) in apps/$project/**
Release CI workflow will create a tag using the format: apps/$project/vX.Y.Z – This is not valid semantic versioning but it is a common approach to tagging multiple components that live within the same repository. – Alternatively, using X.Y.Z-foo or X.Y.Z+bar is indeed a valid semver but it also implies something entirely different than what you intended (for example, - is used for signifying a pre-release; + can be used to specify build metadata).

There are a few gotchas.

Gotcha: Invalid Semver

some/$project/vX.Y.Z is NOT valid semver - if your release consists of creating artifacts and pushing them to other registries (such as to crates.io, NPM, or PyPI), you will DEFINITELY need to strip the prefix some/$project/v before you submit the artifact.

And this brings us to why using pre-release and build tags in semver is NOT going to work — stripping prefixes is common enough; stripping suffixes — not so much.

For example, we use mathieudutour/github-tag-action Github Action to automatically create and/or increase semver based tags. Neat, but the important part is that it has support for tag_prefix - meaning if it’s specified, it will look for tags with the given prefix and strip it when interpreting and bumping the semver.

Secondly, the github-tag-action also exposes outputs that will contain just the new version, without the prefix. This way you can avoid doing hacky sed calls to replace the prefix etc.

Gotcha: Go Pkg Versions

If your monorepo contains Go packages, they are likely stored in some subdir, such as libs/my-go-pkg/*. The problem here is that go mod expects go.mod and go.sum to be at the root of the repo… which is almost definitely not going to be the case with a monorepo.

You can get around this by having the path to the go module specified in the tag in the format of path/to/[email protected] - and this is another reason why choosing the “prefix” approach is a good idea - it works with Go.

For an example of how we got this to work, take a look at the streamdal/streamdal repo and specifically the lib/protos and libs/protos/build/go subdirs.

⚠️ This functionality is known as a “multi-module repository”.⚠️

You can read more about it on the Go Wiki.

Gotcha: Publishing to Deno

One of the release destinations for our protos lib is to deno.land.

The usual workflow involves setting up your Github repo to fire a webhook for deno.land whenever a branch or a tag is created.

The URL for the webhook is: https://api.deno.land/webhook/gh/streamdal_protos

But this won’t work for two reasons:

The module you are trying to publish lives in a subdir
The tags in your repo now have a prefix such as libs/[email protected]

Thankfully, Deno thought ahead of this and the webhook endpoint supports two query params: subdir and tag_prefix.

⚠️ Docs for deno.land module publishing: https://github.com/denoland/deno_registry2/blob/main/API.md

Takeaway

Once you overcome the usual CI-related and organizational hurdles of migrating to a monorepo, you will almost certainly have to deal with versions, tags, or releases.

And as illustrated by the “gotchas” — they will almost certainly involve the tag prefix in some way.

Before looking at other solutions, always look into the possibility of “prefix stripping”.

Good luck!