Buildbot-nix is a NixOS module designed to integrate Buildbot, a continuous integration (CI) framework, into the Nix ecosystem. This module is under active development, and while it's generally stable and widely used, please be aware that some APIs may change over time.
To set up Buildbot using Buildbot-nix, you can start by exploring the provided examples:
Additionally, you can find real-world examples at the end of this document.
Buildbot masters and workers can be deployed either on the same machine or on separate machines. To support multiple architectures, configure them as nix remote builders. For a practical NixOS example, see this remote builder configuration.
buildbot with NixOS 24.05 (stable release)The module applies custom patches that only apply to buildbot > 4.0.0. To use buildbot-nix with NixOS 24.05, you should therefore not override the nixpkgs input to your own stable version of buildbot-nix and leave it to the default instead that is set to nixos-unstable-small.
So instead of using this in your flake
inputs = {
buildbot-nix.url = "github:nix-community/buildbot-nix";
buildbot-nix.inputs.nixpkgs.follows = "nixpkgs";
};Just use:
inputs = {
buildbot-nix.url = "github:nix-community/buildbot-nix";
};An alternative is to point nixpkgs to your own version of nixpkgs-unstable in case you are already using it elsewhere.
Buildbot-nix automatically triggers builds for your project under these conditions:
It does this by evaluating the .#checks attribute of your project's flake in
parallel. Each attribute found results in a separate build step. You can test
these builds locally using nix flake check -L or
nix-fast-build.
If you need to build other parts of your flake, such as packages or NixOS
machines, you should re-export these into the .#checks output. Here are two
examples to guide you:
At the moment buildbot-nix offers two access modes, public and
fullyPrivate. public is the default and gives read-only access to all of
buildbot, including builds, logs and builders. For read-write access,
authentication is still needed, this is controlled by the authBackend option.
fullyPrivate will hide buildbot behind oauth2-proxy which protects the whole
buildbot instance. buildbot fetches the currently authenticated user from
oauth2-proxy so the same admin, organisation rules apply.
fullyPrivate acccess mode is a workaround as buildbot does not support hiding
information natively as now.
For some actions a login is required. This login can either be based on GitHub
or on Gitea (more logins may follow). The backend is set by the
services.buildbot-nix.master.authBackend NixOS option ("gitea"/"github",
"github" by default).
We have the following two roles:
This is the preferred option to setup buildbot-nix for GitHub.
To integrate with GitHub using app authentication:
https://github.com/settings/apps/new for single-user installations or
https://github.com/organizations/<org>/settings/apps/new for
organisations where <org> is the name of your GitHub organizaction.https://buildbot.<your-domain>https://buildbot.<your-domain>/auth/login.services.buildbot-nix.master.github.authType.app.id = <your-github-id>;services.buildbot-nix.master.github.authType.app.secretKeyFile = "/path/to.pem";To integrate with GitHub using legacy token authentication:
admin:repo_hook and repo
permissions. For GitHub organizations, it's advisable to create a separate
GitHub user for managing repository webhooks.https://<your-domain>/auth/login.Afterwards add the configured github topic to every project that should build with buildbot-nix. Notice that the buildbot user needs to have admin access to this repository because it needs to install a webhook.
To integrate with Gitea
write:repository and write:user permission. For Gitea organizations, it's
advisable to create a separate Gitea user. Buildbot-nix will use this token
to automatically setup a webhook in the repository.https://buildbot.your-buildbot-domain.com/auth/login, where
buildbot.your-buildbot-domain.com should be replaced with the actual domain
that your buildbot is running on.Afterwards add the configured gitea topic to every project that should build with buildbot-nix. Notice that the buildbot user needs to have repository write access to this repository because it needs to install a webhook in the repository.
To enable fully private mode, set acessMode.fullyPrivate to an attrset
containing the required options for fully private use, refer to the examples and
module implementation (nix/master.nix).
This access mode honors the admins option in addition to the
accessMode.fullyPrivate.organisations option. To allow access from certain
organisations, you must explicitly list them.
If you've set authBackend previously, unset it, or you will get an error about
a conflicting definitions. fullyPrivate requires the authBackend to be set
to basichttpauth to function (this is handled by the module, which is why you
can leave it unset). For a concrete example please refer to
fully-private-github
Currently buildbot-nix will look for a file named buildbot-nix.toml in the
root of whichever branch it's currently evaluating, parse it as TOML and apply
the configuration specified. The following table illustrates the supported
options.
| key | type | description | default | example | |
|---|---|---|---|---|---|
| lock file | lock_file |
str |
dictates which lock file buildbot-nix will use when evaluating your flake |
flake.lock |
have multiple lockfiles, one for nixpkgs-stable, one for nixpkgs-unstable or by default pin an input to a private repo, but have a lockfile with that private repo replaced by a public repo for CI |
| attribute | attribute |
str |
which attribute in the flake to evaluate and build | checks |
using a different attribute, like hydraJobs |
To access the build results on other machines there are two options at the moment
You can set up a binary cache on your buildbot-worker machine to make its nix store accessible from other machines. Check out the README of the project, for an example configuration
Buildbot-nix also supports pushing packages to cachix. Check out the comment out example configuration in our repository.
Buildbot-nix does not have native support for pushing packages to attic yet. However it's possible to integrate run a systemd service as described in this example configuration. The systemd service watches for changes in the local buildbot-nix store and uploads the contents to the attic cache.
See Buildbot-nix in action in these deployments:
The following instances run on GitHub:
The following instances integrated with Gitea:
We have a matrix channel at buildbot-nix.
For commercial support, please contact Mic92 at joerg@thalheim.io or reach out to Numtide.