Open aluckenbaugh opened 3 years ago
Thanks for the extensive analysis, and sorry you're having this problem.
I assume - but please verify - that a regular docker pull
from a WSL2 or Cygwin shell works?
If that's the case, then honestly I think the second approach is reasonable. In this area, Testcontainers has to mimic the behaviour of Docker's CLI and how it interacts with the credential helpers/stores.
If Docker CLI works then I'd imagine, but haven't checked, that it's automatically adding .cmd
to the command (or maybe trying both with/without .cmd
). We should investigate that and implement the same behaviour in Testcontainers.
A regular docker pull
does work and is currently needed for us to download an image from gcr.io
Only after changing the RegistryAuthLocator as shown above the download works directly with testcontainer.
Yes, as @jglink wrote, a regular docker pull
works for us from the command line with WSL2.
I think we'll probably need to make Teatcontainers try with a .bat
extension. I did check and Docker CLI doesn't have this workaround, which is a little strange.
When you're encountering this problem, are you launching Testcontainers tests in an IDE, perhaps? Is the same true when running tests under a WSL2 shell? This might explain why the extension is necessary, and also why the Docker CLI itself doesn't seem to need it.
Sorry, I think I misspoke earlier: we have been running TestContainers in a Maven project via both IntelliJ and a Windows Terminal / Command Prompt session and have observed the same errors in each. As far as I know our developers haven't been running when inside a WSL2 shell specifically though.
At least with these versions: Google Cloud SDK 340.0.0 bq 2.0.67 core 2021.05.07 gsutil 4.61
the docker-credential-gcloud.cmd
script doesn't have a corresponding *.bat file, so it seems like it would be necessary to use a .cmd extension.
Thanks for clarifying - also thanks for correcting me about the extension.
I think this issue has been fixed via code in master.
I am not sure it was done to directly fix this bug but building and running master with this change has resolved the above issue for me.
Is there a timeline of when new releases are made from master for when this may be available in artifacts?
Problem
If TestContainers detects that an image uploaded to a protected gcr.io docker registry needs to be downloaded it will fail when trying to authenticate using the Google Cloud SDK when running on Windows. At the root it seems this is caused by the
org.testcontainers.utility.RegistryAuthLocator#getCredentialProgramName()
function calculating "docker-credential-gcloud" for the command line to execute, but the actual filename from the google-cloud-sdk is "docker-credential-gcloud.cmd". Therefore an exception like the following is logged and the image pull fails.Please note the directory containing "docker-credential-gcloud.cmd" is indeed on the system PATH.
Logs
10:49:23.379 INFO [🐳 ] ( agroal-11) Pulling docker image: gcr.io/path/to/myimage:latest. Please be patient; this may take some time but only needs to be done once. 10:49:23.380 DEBUG [ or.te.ut.RegistryAuthLocator] ( agroal-11) Looking up auth config for image: gcr.io/path/to/myimage:latest at registry: gcr.io 10:49:23.380 DEBUG [ or.te.ut.RegistryAuthLocator] ( agroal-11) RegistryAuthLocator has configFile: C:\Users\myusername.docker\config.json (exists) and commandPathPrefix: 10:49:23.381 DEBUG [ or.te.ut.RegistryAuthLocator] ( agroal-11) registryName [gcr.io] for dockerImageName [gcr.io/path/to/myimage:latest] 10:49:23.381 DEBUG [ or.te.ut.RegistryAuthLocator] ( agroal-11) Executing docker credential provider: docker-credential-gcloud to locate auth config for: gcr.io 10:49:23.381 DEBUG [ or.te.sh.or.ze.ex.ProcessExecutor] ( agroal-11) Executing [docker-credential-gcloud, get]. 10:49:23.383 DEBUG [ or.te.ut.RegistryAuthLocator] ( agroal-11) Failure running docker credential helper/store (docker-credential-gcloud) 10:49:23.383 INFO [ or.te.ut.RegistryAuthLocator] ( agroal-11) Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: gcr.io/path/to/myimage:latest, configFile: C:\Users\myusername.docker\config.json. Falling back to docker-java default behaviour. Exception message: Could not execute [docker-credential-gcloud, get]. Error=2, The system cannot find the file specified 10:49:23.383 DEBUG [ or.te.ut.RegistryAuthLocator] ( agroal-11) No matching Auth Configs - falling back to defaultAuthConfig [null] 10:49:23.383 DEBUG [or.te.do.AuthDelegatingDockerClientConfi] ( agroal-11) Effective auth config [null] 10:49:24.251 ERROR [ co.gi.do.ap.as.ResultCallbackTemplate] (docker-java-stream--12250) Error during callback: com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication"}
Environment details:
Possible Solutions
gcloud auth configure-docker
and produces a json file like the following:This approach does not seem desirable though since it is a manual change and a regular docker pull for this container works fine.
Update
RegistryAuthLocator
to add the ".cmd" suffix to the command. I'm not sure if this is the best way to fix this issue, but I hacked this together quickly to test it out and it gets us past the problem for the moment. It requires a new member variable:and an update to the protected constructor to initialize this.commandExtension when the OS is windows: