Pulling -esgz images is slower than -org images using docker buildx

[signed-long]

I'm trying to set up a test of pulling pre-converted images with and without lazy-pulling/using the stargz snapshotter.

Doing this in a gha workflow, see the commands run here: https://github.com/signed-long/gha-labs/blob/cd9fb446960c2e2ef812208ca65ec1ab5bac46e5/.github/workflows/stargz-test.yaml#L68

And the logs here: https://github.com/signed-long/gha-labs/actions/runs/9120963253/job/25079331611

The esgz image builds slower than the original image. Is doing these install steps necessary, or just docker buildx create --use --name lazy-builder --buildkitd-flags '--oci-worker-snapshotter=stargz'? The logs do say:

#0 building with "lazy-builder" instance using docker-container driver

Any help with setting up this test is appreciated!

[ktock]

docker buildx build --load -t hello-esgz -f Dockerfile.stargz_esgz .

Why does the result image need to load to Docker? This causes downloading of the result image (including lazy base image) to the node so that they can be sent to Docker. Instead, should this be pushed to the registry? This can prevent downloading layers to the node with reusing existing layers on the registry.

[signed-long]

My end goal is to run some tests inside the container with docker run.

[ktock]

In that case, you also need to enable stargz-snapshotter on Docker as well : https://github.com/containerd/stargz-snapshotter/blob/201c672a8265b3e623f14b1e29868e6c25778def/docs/INSTALL.md#install-stargz-snapshotter-for-dockermoby-with-systemd

[ktock]

--load will still cause downloading of the entire image so you might want to pass the image from builder to the runtime via the registry.

[signed-long]

Thank you for the help, I now see this error when following the install steps from systemctl restart containerd:

https://github.com/signed-long/gha-labs/actions/runs/9122641818/job/25083746461#step:4:865

[ktock]

What error log does containerd process provide?

[signed-long]

here is the containerd log: https://github.com/signed-long/gha-labs/actions/runs/9135332218/job/25122494490#step:5:7

[ktock]

https://github.com/signed-long/gha-labs/actions/runs/9135332218/job/25122494490#step:5:128

Looks like a configuration error happens in the config file. Could you check the contains? The error reports about cri configuration error that looks like unrelated to Docker + lazy pulling configuration.

[signed-long]

Got it, removed the disabled_plugins = ["cri"] from the Runner's /etc/containerd/config.toml file and the containerd service restated successfully.

Can now see running the container directly from the registry is faster with esgz image:

Thanks for the assistance @ktock