nix2container provides an efficient container development workflow with images built by Nix: it doesn't write tarballs to the Nix store and allows to skip already pushed layers (without having to rebuild them).
This is based on ideas developed in this blog post.
{
inputs.nix2container.url = "github:nlewo/nix2container";
outputs = { self, nixpkgs, nix2container }: let
pkgs = import nixpkgs { system = "x86_64-linux"; };
nix2containerPkgs = nix2container.packages.x86_64-linux;
in {
packages.x86_64-linux.hello = nix2containerPkgs.nix2container.buildImage {
name = "hello";
config = {
entrypoint = ["${pkgs.hello}/bin/hello"];
};
};
};
}
This image can then be loaded into Docker with
$ nix run .#hello.copyToDockerDaemon
$ docker run hello:latest
Hello, world!
To load and run the bash
example image into Podman:
$ nix run github:nlewo/nix2container#examples.bash.copyToPodman
$ podman run -it bash
bash
: Bash in /bin/
fromImage
: Alpine as base imagefromImageManifest
: Alpine as base image, from a stored manifest.json
.nginx
nonReproducible
: with a non reproducible store path :/openbar
: set permissions on files (without root nor VM)uwsgi
: isolate dependencies in layerslayered
: build a layered image as described in this blog postnix2container.buildImage
Function arguments are:
name
(required): the name of the image.
tag
(defaults to the image output hash): the tag of the image.
config
(defaults to {}
): an attribute set describing an image configuration as
defined in the OCI image
specification.
copyToRoot
(defaults to null
): a derivation (or list of
derivations) copied in the image root directory (store path
prefixes /nix/store/hash-path
are removed, in order to relocate
them at the image /
).
pkgs.buildEnv
can be used to build a derivation which has to be copied to
the image root. For instance, to get bash and coreutils in the image /bin
:
copyToRoot = pkgs.buildEnv {
name = "root";
paths = [ pkgs.bashInteractive pkgs.coreutils ];
pathsToLink = [ "/bin" ];
};
fromImage
(defaults to null
): an image that is used as base
image of this image; use pullImage
or pullImageFromManifest
to
supply this.
maxLayers
(defaults to 1
): the maximum number of layers to
create. This is based on the store path "popularity" as described
in this blog
post. Note
this is applied on the image layers and not on layers added with
the buildImage.layers
attribute.
perms
(defaults to []
): a list of file permisssions which are
set when the tar layer is created: these permissions are not
written to the Nix store.
Each element of this permission list is a dict such as
{ path = "a store path";
regex = ".*";
mode = "0664";
}
The mode is applied on a specific path. In this path subtree, the mode is then applied on all files matching the regex.
initializeNixDatabase
(defaults to false
): to initialize the
Nix database with all store paths added into the image. Note this
is only useful to run nix commands from the image, for instance to
build an image used by a CI to run Nix builds.
layers
(defaults to []
): a list of layers built with the
buildLayer function: if a store path in deps or contents belongs
to one of these layers, this store path is skipped. This is pretty
useful to isolate store paths that are often updated from more
stable store paths, to speed up build and push time.
nix2container.pullImage
Pull an image from a container registry by name and tag/digest, storing the
entirety of the image (manifest and layer tarballs) in a single store path.
The supplied sha256
is the narhash of that store path.
Function arguments are:
imageName
(required): the name of the image to pull.
imageDigest
(required): the digest of the image to pull.
sha256
(required): the sha256 of the resulting fixed output derivation.
os
(defaults to linux
)
arch
(defaults to x86_64
)
tlsVerify
(defaults to true
)
nix2container.pullImageFromManifest
Pull a base image from a container registry using a supplied manifest file, and the
hashes contained within it. The advantages of this over the basic pullImage
:
manifest.json
from the registry; no need to actually pull the whole
image just to compute a new narhash for it.With this function the manifest.json
acts as a lockfile meant to be stored in
source control alongside the Nix container definitions. As a convenience, the manifest
can be fetched/updated using the supplied passthru script, eg:
nix run .#examples.fromImageManifest.fromImage.getManifest > examples/alpine-manifest.json
Function arguments are:
imageName
(required): the name of the image to pull.
imageManifest
(required): the manifest file of the image to pull.
imageTag
(defaults to latest
)
os
(defaults to linux
)
arch
(defaults to x86_64
)
tlsVerify
(defaults to true
)
registryUrl
(defaults to registry.hub.docker.com
)
Note that imageTag
, os
, and arch
do not affect the pulled image; that is
governed entirely by the supplied manifest.json
file. These arguments are
used for the manifest-selection logic in the included getManifest
script.
If the Nix daemon is used for building, here is how to set up registry authentication.
docker login URL
to whatever it is~/.docker/config.json
to /etc/nix/skopeo/auth.json
nixbld
group:
sudo chmod -R g+rx /etc/nix/skopeo
sudo chgrp -R nixbld /etc/nix/skopeo
extra-sandbox-paths = /etc/skopeo/auth.json=/etc/nix/skopeo/auth.json
Every time a new registry authentication has to be added, update
/etc/nix/skopeo/auth.json
file.
nix2container.buildLayer
For most use cases, this function is not required. However, it could be
useful to explicitly isolate some parts of the image in dedicated
layers, for caching (see the "Isolate dependencies in dedicated
layers" section) or non reproducibility (see the reproducible
argument) purposes.
Function arguments are:
deps
(defaults to []
): a list of store paths to include in the
layer.
copyToRoot
(defaults to null
): a derivation (or list of
derivations) copied in the image root directory (store path
prefixes /nix/store/hash-path
are removed, in order to relocate
them at the image /
).
pkgs.buildEnv
can be used to build a derivation which has to be copied to
the image root. For instance, to get bash and coreutils in the image /bin
:
copyToRoot = pkgs.buildEnv {
name = "root";
paths = [ pkgs.bashInteractive pkgs.coreutils ];
pathsToLink = [ "/bin" ];
};
reproducible
(defaults to true
): If false
, the layer tarball
is stored in the store path. This is useful when the layer
dependencies are not bit reproducible: it allows to have the layer
tarball and its hash in the same store path.
maxLayers
(defaults to 1
): the maximum number of layers to
create. This is based on the store path "popularity" as described
in this blog
post. Note
this is applied on the image layers and not on layers added with
the buildLayer.layers
attribute.
perms
(defaults to []
): a list of file permisssions which are
set when the tar layer is created: these permissions are not
written to the Nix store.
Each element of this permission list is a dict such as
{ path = "a store path";
regex = ".*";
mode = "0664";
}
The mode is applied on a specific path. In this path subtree, the mode is then applied on all files matching the regex.
layers
(defaults to []
): a list of layers built with the
buildLayer
function: if a store path in deps or contents belongs
to one of these layers, this store path is skipped. This is pretty
useful to isolate store paths that are often updated from more
stable store paths, to speed up build and push time.
ignore
(defaults to null
): a store path to ignore when
building the layer. This is mainly useful to ignore the
configuration file from the container layer.
It is possible to isolate application dependencies in a dedicated layer. This layer is built by its own derivation: if storepaths composing this layer don't change, the layer is not rebuilt. Moreover, Skopeo can avoid to push this layer if it has already been pushed.
Let's consider an application
printing a conversation. This script
depends on bash
and the hello
binary. Because most of the changes
concern the script itself, it would be nice to isolate scripts
dependencies in a dedicated layer: when we modify the script, we only
need to rebuild and push the layer containing the script. The layer
containing dependencies won't be rebuilt and pushed.
As shown below, the buildImage.layers
attribute allows to
explicitly specify a set of dependencies to isolate.
{ pkgs }:
let
application = pkgs.writeScript "conversation" ''
${pkgs.hello}/bin/hello
echo "Haaa aa... I'm dying!!!"
'';
in
pkgs.nix2container.buildImage {
name = "hello";
config = {
entrypoint = ["${pkgs.bash}/bin/bash" application];
};
layers = [
(pkgs.nix2container.buildLayer { deps = [pkgs.bash pkgs.hello]; })
];
}
This image contains 2 layers: a layer with bash
and hello
closures
and a second layer containing the script only.
In real life, the isolated layer can contains a Python environment or Node modules.
See Nix & Docker: Layer explicitly without duplicate packages! for learning how to avoid duplicate store paths in your explicitly layered images.
The main goal of nix2container is to provide fast rebuild/push
container cycles. In the following, we provide an order of magnitude
of rebuild and repush time, for the uwsgi
image.
warning: this is quick and dirty benchmarks which only provide an order of magnitude
We build the container and push the container. We then made a small
change in the hello.py
file to trigger a rebuild and a push.
Method | Rebuild/repush time | Executed command |
---|---|---|
nix2container.buildImage | ~1.8s | nix run .#example.uwsgi.copyToRegistry |
dockerTools.streamLayeredImage | ~7.5s | nix build .#example.uwsgi \| docker load |
dockerTools.buildImage | ~10s | nix build .#example.uwsgi; skopeo copy docker-archive://./result docker://localhost:5000/uwsgi:latest |
Note we could not compare the same distribution mechanisms because
nix run .#tests.all
This builds several example images with Nix, loads them with Skopeo, runs them with Podman, and test output logs.
Not that, unfortunately, these tests are not executed in the Nix sandbox because it is currently not possible to run a container in the Nix sandbox.
It is also possible to run a specific test:
nix run .#tests.basic
This library is currently used by the Skopeo nix
transport available
in this branch.
For more information, refer to the Go documentation.