search

containerd / stargz-snapshotter

Fast container image distribution plugin with lazy pulling
https://github.com/containerd/containerd/issues/3731
Apache License 2.0
1.09k stars 109 forks source link

Is it possible to lazy pull eStargz images during (docker buildx) build #1390

Open dmosdallas opened 9 months ago

dmosdallas commented 9 months ago

Background

I am attempting to build new images that are based upon images that are already in the eStargz format.

With docker buildx, I can run the following:

docker buildx build -t $IMAGE_NAME \
    -o type=registry,oci-mediatypes=true,compression=estargz,force-compression=true \
    .

Problem

The above successfully builds a new eStargz-formatted image, but it fully pulls the eStargz-formatted base image during the build.

Ideas

I've come across some buildkit documentation that describes how to enable lazy pulling of images using buildkitd & buildctl directly.

It is probably my lack of understanding, but I've gotten a bit lost in this documentation. As far as I can tell, a vanilla install of docker does not include buildkitd and I'm not totally sure how buildx relates to it -- buildx seems to be a standalone plugin that exposes some features of buildkit without buildkitd?

Question

Is there a way to configure the docker buildx plugin to use the --oci-worker-snapshotter=stargz as described in the above docs, or am I totally misunderstanding how this is wired together?

ktock commented 9 months ago

@dmosdallas

Is there a way to configure the docker buildx plugin to use the --oci-worker-snapshotter=stargz as described in the above docs, or am I totally misunderstanding how this is wired together?

Please try "BuildKit" section of https://medium.com/nttlabs/lazy-pulling-estargz-ef35812d73de

$ docker buildx create --use --name lazy-builder --buildkitd-flags '--oci-worker-snapshotter=stargz' $ docker buildx inspect --bootstrap lazy-builder

dmosdallas commented 9 months ago

Perfect, thanks for the pointer!

dmosdallas commented 9 months ago

Okay, I've had a chance to test this out and I think I'm running into the same credential issue mentioned in https://github.com/containerd/stargz-snapshotter/issues/1389#issuecomment-1725949847.

When I use the lazy-builder, it's running a buildkitd instance in a docker container which doesn't have access to the credentials for my private repo.

Is there a way to propagate docker credentials to the instance of buildkitd running in the container created with a docker buildx create command?

ktock commented 9 months ago

@dmosdallas stargz-snapshotter is embedded to buildktid and it receives creds via docker command over buildkitd API so I believe https://github.com/containerd/stargz-snapshotter/issues/1389#issuecomment-1725949847 should not occur if you use --oci-worker-snapshotter=stargz. If it doesn't work, please post reproduce steps so that we can work on fixing that.