`docker build` can't retrieve creds from OSX keychain that `docker login` puts in

[fajpunk]

Expected behavior

Docker tries to retrieve the same creds from the OSX keychain on build as it puts into the keychain on login, build succeeds.

Actual behavior

Docker passes a different URL to the credentials helper for the login command than it does for the buildcommand, build fails with unauthorized: authentication required

Information

macOS: version 10.11.6 (build: 15G1108)
logs: /tmp/15DC209E-3B90-497B-9C38-735430343F71/20170317-104413.tar.gz
[OK]     vmnetd
[OK]     dns
[OK]     driver.amd64-linux
[OK]     virtualization VT-X
[OK]     app
[OK]     moby
[OK]     system
[OK]     moby-syslog
[OK]     db
[OK]     env
[OK]     virtualization kern.hv_support
[OK]     slirp
[OK]     osxfs
[OK]     moby-console
[OK]     logs
[OK]     docker-cli
[OK]     menubar
[OK]     disk

Steps to reproduce the behavior

1. docker login -u AWS -p mytoken -e none https://myaccountid.dkr.ecr.us-east-1.amazonaws.com
2. docker build with a Dockerfile that starts like: FROM myaccountid.dkr.ecr.us-east-1.amazonaws.com/appbase/python2.7.5

More information

aws-cli/1.11.61 Python/3.5.0 Darwin/15.6.0 botocore/1.5.24

This is affecting the awscli aws ecr get-login command on OSX.

Running this AWS CLI command gives this docker login command:

$ aws ecr get-login --profile myprofile --region us-east-1
docker login -u AWS -p mytoken -e none https://myaccountid.dkr.ecr.us-east-1.amazonaws.com

Running that docker login command puts an entry into the osx keychain with the Where set to:

http:///myaccountid.dk.er.us-east-1.amazonaws.com

Note the incorrect protocol (http instead of https) and the three slashes after the protocol, as previous comments have described.

Then, when I try to build an image from an image in that registry, with a FROM line like this:

FROM myaccountid.dkr.ecr.us-east-1.amazonaws.com/appbase/python2.7.5

I get:

/usr/local/bin/docker build --rm -t myaccountid.dkr.ecr.us-east-1.amazonaws.com/myimage -f Dockerfile .
Sending build context to Docker daemon 1.045 MB
Step 1/11 : FROM myaccountid.dkr.ecr.us-east-1.amazonaws.com/myimage
Pulling repository myaccountid.dkr.ecr.us-east-1.amazonaws.com/myimage
unauthorized: authentication required

But if I manually edit the osx keychain entry and remove that third /, then that build succeeds. So presumably, that third slash is making it so that the keychain entry containing the login token can't be found.

Code

Docker-credential-osxkeychain expects the ServerURL value to be prefixed with a protocol (http:// or https://), but for the login command, docker passes just a domain, no protocol. Docker is stripping the protocol here: https://github.com/docker/docker/blob/master/cli/command/registry.go#L82

Because of this, splitServer (https://github.com/docker/docker-credential-helpers/blob/master/osxkeychain/osxkeychain_darwin.go#L137) is not parsing it correctly into the struct that gets passed to the OS command to add the entry to the keychain.

Specifically, u.Scheme will be blank here (https://github.com/docker/docker-credential-helpers/blob/master/osxkeychain/osxkeychain_darwin.go#L155), so will get defaulted to http, and because there's no protocol, url.Parse is putting the domain into u.Path, which I think explains the extra slash in the keychain entry.

On build however, docker is appending http:// to the registry, which gets parsed through the same splitServer function, which returns a struct that doesn't match the struct used to put the login creds in the keychain.

Potential solution

I've put up a PR for a potential change to the docker-credential-osxkeychain program: https://github.com/docker/docker-credential-helpers/pull/55

[djs55]

Thanks for the report, analysis and proposed fix! I'll set the state of this ticket to acknowledged and will wait for the maintainers of docker-credential-helpers to respond.

[kockas]

Actually, this is just fragment of the issue. Depends, which command is used.

For my version: 17.03.1-ce:

docker login some_web.com creates in Keychain credentials with field Where with value http://:

• This does not work with docker build
• It works with docker pull some_web.com/image_name.

When you fix field Where and put there https://some_web.com:

• This does work for docker build
• This stop working for docker pull some_web.com/image_name.

So I believe, there is quite mess in it. Any idea, when this gonna be fixed? Or should I get back to older version?

[fajpunk]

Do I need to do anything with this issue or the associated PR given the Moby switch?

[aniruthmp]

Hey, following @kockas (my version is 17.03.1-ce) I updated my keychain to https://some_web.com and I was able to successfully run docker build -f Dockerfile But spotify/docker-maven-plugin fails

[fajpunk]

@kockas, this PR should normalize all access to the creds in the keychain, which should allow both of your usecases to function correctly: https://github.com/docker/docker-credential-helpers/pull/55. I may be missing something though, if you see a reason that that PR won't address it, let me/us know for sure!

[kockas]

@fajpunk I'm not able to properly review PR, however I can test locally, if you provide me with binary, and 'how to' instructions. The only thing I can say now, is that I was able to use pull and build simultaneously if I put two entries into keychain. Once it was http:// and once it was fixed url eg. https://some_web.com First one worked for build, and second for pull. :)

[pwagland]

What is the current status of this issue? Will it be fixed in an upcoming release?

[jluethi]

Yes, a fix for this would be very much appreciated. I'm currently using a (very ugly) workaround, but maybe it's helpful for others: Logging in to docker on a different machine (e.g. an ubuntu VM) still gives me that auth token in the ~/.docker/config.json file. When I copy the config.json file to my local machine, I can then use docker normally again (until I log out). I therefore just keep a copy of the config.json file around to "manually log in" to docker.

[briceburg]

What's interesting is that docker pull works (successfully uses creds from buildchain) -- but docker build does not. So for instance if your Dockerfile looks like

FROM registry.acme.com/image:latest
...

you could do a workaround where you pull first:

docker pull registry.acme.com/image:latest
docker build .

keep in mind pulling during the build will always fail, e.g.

docker build --pull .

[briceburg]

A full test looks like;

Dockerfile;

FROM registry.acme.com/image:latest
...

commands;

# 1. prepare (authenticate and purge registry image from cache) 
docker login registry.acme.com
docker rmi --force registry.acme.com/image:latest

# 2. test
docker pull registry.acme.com
docker build
## ^^^ works!

docker build --pull .
## ^^^ fails w/ unauthorized

[tim-gp]

Confirmed that changing the Where in the appropriate Docker Credentials from http to https makes build work but does break pull.

[jeremie-olivier]

When using the Edge ( not stable ) version, you can choose to desable storing docker login in OSX Keychain. Everything now works well for me

[pwagland]

It seems that the latest edge release works. I had to manually delete the credentials from my keychain, and relogin, but after that, building worked again.

[n4ss]

We fixed this in edge release, you'll indeed have to make the mentioned steps from ^^^ manually.

[docker-robott]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[Labutin]

This is a terrible issue! Any progress?

[aniruthmp]

/lifecycle frozen

[WattsInABox]

This does not appear to be fixed in Docker desktop 2.2.0 so I'm assuming this isn't an edge thing given the time span here. Is this an issue with Desktop or with Docker itself? Is there any work around besides moving away from the osx keychain provider?

[WattsInABox]

Actually my current problem appears to be with the os x keychain provider itself. I was able to verify that no matter what you do, it prepends "https" to the ServerURL. Since other tools (compose, dobi, etc.) don't expect https to be there, they cannot find the credentials for the site.

[jblachly]

Still a bug in 4.15.0 (93002)

~/.docker/config.json includes repository without https:// -> xxxxxxxxxxxx.dkr.ecr.us-east-2.amazonaws.com

Keychain prepends https://

I also deleted the Keychain item (which was of Kind "Internet password") and recreated it as Kind "application password", which does not prepend https:// ; Docker Desktop still does not query Keychain AFAICT.

However, Docker Desktop Mac <--> Keychain DOES WORK for ghcr.io, which is strange