Using images while offline fails even if they are cached

[sipsma]

If you run a pipeline that references an image using a tag, the tag needs to be resolved to a SHA, which requires internet access (or, more specifically, network access to the registry).

This means that if you are offline you can't run those pipelines, even if the images are cached locally. This also results in the inability to run pretty much all of our test suite while offline, which is quite annoying if you are working on dagger and only have intermittent internet access.

Workarounds possible now:

1. Specify each image using its fully resolved SHA, in which case the network call should be avoided
   • This needs to be verified. I think it's true in theory, but not 100% sure whether it also is in practice
2. Setup a local registry mirror that caches everything you need, configure your engine to use that mirror
   • Possible, and this is actually what buildkit's integration tests do, but a fair bit of overhead.

To address this, we should decide between these two options (or more if there's more ideas):

1. Add support for Buildkit's ResolveModePreferLocal setting, which results in Buildkit checking cache for a given tag first before resolving it to a SHA
   • I don't think we want to just make this the default because in a lot of cases it's not what you want. e.g. If you are specifying go:1.20, that tag gets updated for every one of go's patch releases, but if you always prefer local then you'll never pick those up. I suppose :latest is the even more extreme example here.
   • This could be an option to container.from or also a more "global" setting to the session so that you don't have to explicitly provide it to every single container.from (especially nice if you are calling out to pipelines in a different function/package, etc.)
2. Don't address it, just go with the workarounds

[grouville]

This might have an impact on the analytics collection. cc @gerhard

[gerhard]

Yes, this makes a lot of sense.

We are already doing this for the Engine image:

failed to resolve image digest: Get "https://registry.dagger.io/v2/": dial tcp: lookup registry.dagger.io: no such host
falling back to leftover engine
Connected to engine fce67ef5b2b5

I just confirmed that sha256 image references uses the locally cached layers and works without an internet connection:

x ping -t 1 -c 1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: sendto: No route to host

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss

x mage image:runtime
failed to resolve image digest: Get "https://registry.dagger.io/v2/": dial tcp: lookup registry.dagger.io: no such host
falling back to leftover engine
Connected to engine fce67ef5b2b5

#1 resolve image config for docker.io/hexpm/elixir@sha256:c8ea7aa44b215364012b297ef1deb8ca0af460607493c2371f343ca7fa60acf1

...

#1 DONE 0.0s

#2 from hexpm/elixir@sha256:c8ea7aa44b215364012b297ef1deb8ca0af460607493c2371f343ca7fa60acf1
#2 resolve docker.io/hexpm/elixir@sha256:c8ea7aa44b215364012b297ef1deb8ca0af460607493c2371f343ca7fa60acf1 done
#2 DONE 0.0s

#3 📦 RUNTIME IMAGE
#3 DONE 0.0s

The easiest thing would be to have a --local flag which explicitly enables the locally cached images. I can imagine this being useful in other scenarios, such as builds which must be produced with local data only, and nothing from the internet.

WDYT @sipsma?

[matiasinsaurralde]

@gerhard Should this benefit tests execution too? Tests that share some Alpine image should reuse that, etc.