Open sigwinch28 opened 7 months ago
Hmm. At a quick guess, the BuildKit-internal HTTP code is not seeing the proxy config; it should be pulling it from the environment from a brief skim of the code, and I expect the env.https_proxy
driver option to have configured the environment correctly.
It works when cloned locally because the image pull is not done by BuildKit directly, but by a request to containerd, so clearly it is seeing the right proxy config, and the issue lies on the BuildKit side somewhere.
#4725 is a similar issue, as it hits the same failure when trying to talk to auth.docker.io from BuildKit code, but not when underlying containerd pulls an image from a different registry (that doesn't require auth), which aligns with this hypothesis.
Edit: On closer examination, we shell out to git
, so it might actually be unrelated to #4725, but just a flat-out bug that we don't pass HTTP proxy info into that shell-out's environment, and didn't in 0.12.5 either.
but just a flat-out bug that we don't pass HTTP proxy info into that shell-out's environment, and didn't in 0.12.5 either.
Preface: I know nothing about docker / moby internals.
Is it valid to pass the proxy information through in the place you have linked?
If so I'll probably try to test it locally and make a PR; this isn't a hard-requirement for my org's use of docker; but it would be nice to gain the git-related benefits across our images (less information to download across multiple versions since only the git deltas/packs need to download).
Note: I don't have the repo locally in front of me, so this is based on browsing in GitHub. So I may have mistaken linkages here.
The source I linked to should be the right place to expose the HTTP proxy information to Git; it should be handled somewhat like the HTTP auth config, I think, e.g., https://github.com/moby/buildkit/blob/efcde2fcfbc4299ad8f07727fd65562409608c88/client/llb/source.go#L290-L295, provided by a new llb.GitOption
passed into llb.Git
, something like WithGitHTTPProxy(http_proxy, https_proxy, all_proxy, no_proxy)
.
For git sources, the call to llb.Git
is in DetectGitContext
, in this case it's going to be similar to how BUILDKIT_CONTEXT_KEEP_GIT_DIR
is handled, i.e. with a bit of luck the settings are naturally in opts
and can be easily passed into DetectGitContext
. And adjacently, DetectHTTPContext
doesn't appear to get any HTTP proxy settings either, so it might make sense to handle this consistently here. (I suspect that if http sources already work, it's because llb.HTTP
inherits from the proxy settings in the environment rather than from the build-args.)
For copy
/add
git sources, we should get the values from the proxyEnv
extracted from the build-args, which is currently only provided to the "Run" operation, but for this case will need to be provided to the "Copy" operation. That will take care of Copy, but not the actual use-case here, which is for a Git source for the build. (I haven't checked for copy/add http sources, they might also currently work from execution environment instead of build-args.)
A fallback hack for all of the above would be to make llb.Git
work like llb.HTTP
appears to, and grab the http_proxy
etc. env-vars directly in that function, ignoring the build-args passed to the client, and relying on the buildkit server being configured with the correct proxy environment. (Assuming I'm correct that this is how llb.HTTP
works now...)
I can't seem to use an HTTPS git source from behind a corporate proxy, e.g.
http://proxy.contoso.com:8080
.Buildx version:
github.com/docker/buildx v0.12.1 30feaa1a915b869ebc2eea6328624b49facd4bfb
Buildkit runner creation command:buildkit.toml
:Build command:
I have veirifed that
github.com
is not in the$no_proxy
env var.Build output:
logs from build container:
Also, if I provide a branch, it fails too:
The logs show:
however, running a
git clone
in the build container viadocker exec
works fine.It seems that either:
Is this a known issue or an unsupported use case? or maybe some nonobvious missing config?