✨ Import image to local container runtime

[jedevc]

The CLI should support loading images into the local container image store - whether that's docker, containerd through ctr/nerdctl, podman, etc.

Today, this might look something like:

$ dagger call container-echo --string-arg=foo -o ./image.tar
$ docker load -i ./image.tar

However, we should really have a native way to do this - something like:

$ dagger call container-echo --string-arg=foo --load <image>:<tag>

Why?

• Makes it easier for dagger to exist as a full replacement for docker build, as in @verdverm's use case illustrated in discord https://discord.com/channels/707636530424053791/1265384148856471622/1265384148856471622
• We want to avoid users feeling the need to wrap dagger using another cli tool as much as possible - not having an integrated way to do this common task is sad.
• The current approach is slow - even if we have a containerd content API which avoids uploading layers that weren't changed, we can't really take advantage of it (currently, not exposed in dockerd, but somewhere on the roadmap see https://github.com/moby/moby/issues/44369)

So how would this work?

• The CLI should have support for a generic "loader" interface - these take a core container object, and somehow "load" them. We can implement two loaders to start:
  • docker load (the default)
  • Containerd content store streaming, implemented using the session attachables that buildkit itself has - we can allow forwarding a local containerd socket api, or opening a local content store (as the oci export in buildkit uses).
• The syntax could then look something like:
  • dagger call container-echo --string-arg=foo --load <image>:<tag> to load into the default docker store
  • dagger call container-echo --string-arg=foo --load ./path/to/oci/layout/<image>:<tag> to export to a local oci layout
  • dagger call container-echo --string-arg=foo --load ./path/to/containerd/socket to load to the containerd store accessible via that API

Related:

• https://github.com/dagger/dagger/issues/6271

[ghost]

Is this also related to #6999? I had a similar use case to #7368, but ultimately didn't manage to get it working as expected.

Even with the described current practice:

$ dagger call container-echo --string-arg=foo -o ./image.tar
$ docker load -i ./image.tar

I'd run into this issue where the image name and tag would not be preserved when loading the tarball. The export method of container didn't seem to behave like docker image save.

[verdverm]

Forgot about the tagging part, for context, here is the code I use in one project

// writes images into docker daemon on host
export default async function build(cfg: Record<string,any>, client: Client) {
    // Dagger Containers
    const imgs = await images(cfg, client)

    const output = async (img: Container, name: string) => {
        const fullName = `${name}:dirty`
        const fullTar = `/tmp/${fullName}.tar`

        // write tar file, triggers actual image building
        await img.export(fullTar)

        // load into docker
        const { stdout } = await exec(`docker load -i ${fullTar}`)
        const sha = stdout.split(":")
        const hash = sha[sha.length-1].trim()

        // tag image
        await exec(`docker tag ${hash} ts/${fullName}`)

        // cleanup tar file
        await exec(`rm ${fullTar}`)
    }

        // doing this in parallel will make your computer warm...
    await Promise.all(cfg.components.map(async (c) => {
        await output(await imgs[c], cfg[c].name)
    }))
}

[ghost]

Thanks for sharing, that's nifty.

[VasiliyS]

That would be a much welcomed feature!

In my case, I'd like to automate the container workflow for local dev and testing for a library I'm developing. In a nutshell, the library is a client which interacts with a server. This server is under active development and therefore a functionality is needed to automate bootstrapping dev/test/demo environment against various git refs.

Also, the users of the library will benefit from the ability to have all bootstrapping done for them for learning and development.

Currently, I'm exporting and loading a tarball, but it's slow and feels kind of wasteful.

BTW, to deal with tagging, one can simply annotate the container with io.containerd.image.name prior to asTarball call. docker load -i picks up the annotation, including the tag.