search

docker / compose

Define and run multi-container applications with Docker
https://docs.docker.com/compose/
Apache License 2.0
33.97k stars 5.22k forks source link

docker-compose pull fails with error image not found for private repos #1622

Closed kyndtp closed 8 years ago

kyndtp commented 9 years ago

I have just set up a new cloud server with Ubuntu 14.04.2, Docker 1.7.0 and Docker Compose 1.2.0.

When I am executing, docker-compose up -d, I am getting the following errors (the image is coming from a private repo)

Traceback (most recent call last): File "", line 3, in File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 31, in main File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 27, in dispatch File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 59, in perform_command File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 464, in up File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 208, in up File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 214, in recreate_containers File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 199, in create_container File "/code/build/docker-compose/out00-PYZ.pyz/compose.progress_stream", line 37, in stream_output File "/code/build/docker-compose/out00-PYZ.pyz/compose.progress_stream", line 50, in print_output_event compose.progress_stream.StreamOutputError: Error: image togger/main-db:2.1.2 not found

any help would be greatly appreciated

approxit commented 9 years ago

Have you tried using docker login before pulling private image?

kyndtp commented 9 years ago

Hi approxit, yes was logged into the private docker repo, I had checked with docker info, and the username was of the repo account. I am looking at the docker version of a previous setup and the docker version was 1.6.3, am wondering if I have missed something from docker-compose.yml file or need to upgrade docker-compose to 1.3.0

approxit commented 9 years ago

Upgrading to latest version is always worth try. Remember to check 1.3 migrations notes before you make something funky.

Have you tried manual pulling your image? When it fails, it's not a docker-compose problem then.

kyndtp commented 9 years ago

A manual docker pull, worked. I will upgrade to 1.3 and see if this resolves it.

volcomism commented 9 years ago

Docker 1.7 changed it's auth path. Revert back to Docker 1.6 in the meantime until Compose releases 1.3.2.

See issue: https://github.com/docker/compose/issues/1590

nottrobin commented 9 years ago

I'm still getting this issue with more recent versions:

$ docker-compose -v
docker-compose version: 1.3.3
CPython version: 2.7.6
OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
$ docker --version
Docker version 1.7.1, build 786b29d

The problem looks like this:

$ docker-compose up
Building web...
Step 0 : FROM my-user/private-imag
Pulling repository my-user/private-imag
Service 'web' failed to build: Error: image my-user/private-image:latest not found

As reported above, doing docker pull first does work.

nottrobin commented 9 years ago

Also still fails with 1.4.0rc2 (and Docker 1.7.1)

screwgoth commented 9 years ago

I'm facing a similar issue with: $ docker-compose -v docker-compose version: 1.3.3 CPython version: 2.7.9 OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013

$ docker -v Docker version 1.7.1, build 786b29d

Traceback (most recent call last): File "", line 3, in File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 32, in main File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 34, in dispatch File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 66, in perform_command File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 471, in up File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 245, in up File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 318, in execute_convergence_plan File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 212, in create_container File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 243, in ensure_image_exists File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 713, in pull File "/code/build/docker-compose/out00-PYZ.pyz/docker.client", line 590, in pull File "/code/build/docker-compose/out00-PYZ.pyz/docker.auth.auth", line 60, in resolve_repository_name File "/code/build/docker-compose/out00-PYZ.pyz/docker.auth.auth", line 39, in expand_registry_url docker.errors.DockerException: HTTPS endpoint unresponsive and insecure mode isn't enabled.

aanand commented 9 years ago

@screwgoth Pass in --allow-insecure-ssl or upgrade to 1.4.0.

screwgoth commented 9 years ago

Now working with --allow-insecure-ssl. Have to pull the images beforehand for docker-compose up or scale to work. Will try upgrading to 1.4.0 and let you know

Ducatel commented 9 years ago

Same trouble with docker-compose 1.4.0 --allow-insecure-ssl is deprecated now and do nothing

>>> docker-compose version
docker-compose version: 1.4.0
docker-py version: 1.3.1
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013

>>> docker version
Client:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:35:49 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:35:49 UTC 2015
 OS/Arch:      linux/amd64

So still impossible to make a docker-compose pull if an image come from a private repo.

michaeljs1990 commented 9 years ago

Just ran into this today... did a dirty work around and made a makefile that pulls all the images first for me. Not optimal but it works for now. Hopefully it will be fixed in 1.5.

brycereynolds commented 9 years ago

I am encountering the same thing. With 6 - 7 private repos it gets painful :)

Any update on the status of this one?

macropin commented 9 years ago

I just encountered this with docker-compose 1.2.0 when pulling images from quay.io. But it works fine with 1.5.0.

Check that you don't have an old version of docker-compose installed. In my case I had a manual install in ~/bin/.

kenhuang commented 8 years ago

We had issue try to pull private image using docker-compose 1.2.0, after upgrade to 1.5.1, it is working fine using image name "myrepo/myapp" name pattern, we had found another issue that it fail on using "docker.io/myrepo/my-app" as the image name.

We are try to make our scripts be able to switch between "docker.io" and our own docker proxy repo.

aanand commented 8 years ago

@kenhuang I can't reproduce with Compose 1.5.2:

test:
  image: docker.io/tianon/true
$ docker-compose up
Pulling test (docker.io/tianon/true:latest)...
latest: Pulling from tianon/true
f947739ce3ee: Pull complete
bce7ccaa26ca: Pull complete
Digest: sha256:92e44b3024657b093b592ec464ed3c20076804a0346efb99bef2b16cbd8f2460
Status: Downloaded newer image for tianon/true:latest
Creating dockerio_test_1
Attaching to dockerio_test_1
dockerio_test_1 exited with code 0
kenhuang commented 8 years ago

@aanand "docker.io/tianon/true" is a public image, can you try a private image?

aanand commented 8 years ago

OK, I now get what I think you're seeing - docker-compose pull fails, while docker pull aanand/true succeeds. It's a docker-py bug - I'm investigating now.

$ docker-compose pull
Pulling test (docker.io/aanand/true:latest)...
Pulling repository docker.io/aanand/true
ERROR: Error: image aanand/true:latest not found

$ docker pull docker.io/aanand/true
Using default tag: latest
latest: Pulling from aanand/true
Digest: sha256:70ea24b18c7563bf5726476856f8a48e1047390d41d7147c4e0988e7fea5f1b4
Status: Downloaded newer image for aanand/true:latest
aanand commented 8 years ago

I've got a fix in https://github.com/docker/docker-py/pull/874.

aanand commented 8 years ago

Fix has been merged, so the docker.io/<username>/<repo> fix is just blocking on a new docker-py version.

Is anyone continuing to experience the original problem with Compose 1.5? To be specific, the issue is a docker-compose pull failing where a docker pull succeeds.

If so, please paste the output of docker-compose --verbose pull.

gnarea commented 8 years ago

@aanand, Docker Compose 1.5.1:

compose.config.config.find: Using configuration files: ./services.yml
docker.auth.auth.load_config: Trying /home/PROJECT_NAME/.docker/config.json
docker.auth.auth.load_config: File doesn't exist
docker.auth.auth.load_config: Trying /home/PROJECT_NAME/.dockercfg
docker.auth.auth.load_config: Attempting to parse as JSON
docker.auth.auth.load_config: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128)
docker.auth.auth.load_config: Attempting to parse legacy auth file format
docker.auth.auth.load_config: list index out of range
docker.auth.auth.load_config: All parsing attempts failed - returning empty config
compose.cli.command.get_client: Compose version 1.5.1
compose.cli.command.get_client: Docker base_url: http+docker://localunixsocket
compose.cli.command.get_client: Docker version: KernelVersion=3.16.0-0.bpo.4-amd64, Os=linux, BuildTime=Fri Nov 20 17:56:04 UTC 2015, ApiVersion=1.21, Version=1.9.1, GitCommit=a34a1d5, Arch=amd64, GoVersion=go1.4.3
compose.service.pull: Pulling db (postgres:9.4.5)...
compose.cli.verbose_proxy.proxy_callable: docker pull <- (u'postgres', tag=u'9.4.5', stream=True)
docker.api.image.pull: Looking for auth config
docker.api.image.pull: No auth config in memory - loading from filesystem
docker.auth.auth.load_config: Trying /home/PROJECT_NAME/.docker/config.json
docker.auth.auth.load_config: File doesn't exist
docker.auth.auth.load_config: Trying /home/PROJECT_NAME/.dockercfg
docker.auth.auth.load_config: Attempting to parse as JSON
docker.auth.auth.load_config: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128)
docker.auth.auth.load_config: Attempting to parse legacy auth file format
docker.auth.auth.load_config: input() already active
docker.auth.auth.load_config: All parsing attempts failed - returning empty config
docker.auth.auth.resolve_authconfig: Looking for auth entry for 'index.docker.io'
docker.auth.auth.resolve_authconfig: No entry found
docker.api.image.pull: No auth config found
compose.cli.verbose_proxy.proxy_callable: docker pull -> <generator object _stream_helper at 0x7f23f2a5a190>
9.4.5: Pulling from library/postgres
Digest: sha256:10ac746a6b38904320c84b7b4dc8c311fe40aa891701db4c818a5ec68194812e
Status: Image is up to date for postgres:9.4.5
compose.service.pull: Pulling twod-masker (2degrees/PROJECT_NAME:latest)...
compose.cli.verbose_proxy.proxy_callable: docker pull <- ('2degrees/PROJECT_NAME', tag=u'latest', stream=True)
docker.api.image.pull: Looking for auth config
docker.api.image.pull: No auth config in memory - loading from filesystem
docker.auth.auth.load_config: Trying /home/PROJECT_NAME/.docker/config.json
docker.auth.auth.load_config: File doesn't exist
docker.auth.auth.load_config: Trying /home/PROJECT_NAME/.dockercfg
docker.auth.auth.load_config: Attempting to parse as JSON
docker.auth.auth.load_config: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128)
docker.auth.auth.load_config: Attempting to parse legacy auth file format
docker.auth.auth.load_config: input() already active
docker.auth.auth.load_config: All parsing attempts failed - returning empty config
docker.auth.auth.resolve_authconfig: Looking for auth entry for 'index.docker.io'
docker.auth.auth.resolve_authconfig: No entry found
docker.api.image.pull: No auth config found
compose.cli.verbose_proxy.proxy_callable: docker pull -> <generator object _stream_helper at 0x7f23f29f9c30>
Pulling repository docker.io/2degrees/PROJECT_NAME
ERROR: compose.cli.main.main: Error: image 2degrees/PROJECT_NAME:latest not found
macropin commented 8 years ago

@gnarea I think this is your problem... Pulling repository docker.io/2degrees/PROJECT_NAME

gnarea commented 8 years ago

Sorry, I forgot to say that I replaced the actual repository name with "PROJECT_NAME". I'm a bit too paranoid. :-) On 17 Dec 2015 22:42, "Andrew Cutler" notifications@github.com wrote:

@gnarea https://github.com/gnarea I think this is your problem... Pulling repository docker.io/2degrees/PROJECT_NAME

— Reply to this email directly or view it on GitHub https://github.com/docker/compose/issues/1622#issuecomment-165605000.

sascha-egerer commented 8 years ago

I've got the same problem. On some machines docker-compose pull works but on others it does not.

The problem onyl occurs with a private registry. Maybe some parsing problem beause of the : at the port definition? Does that internally make problems as the part behind the : is used as version?

With docker pull everything works fine.

$ docker-compose --version
docker-compose version: 1.5.1
$ docker --version
Docker version 1.9.1, build a34a1d5
$ docker-compose --verbose pull
compose.config.config.find: Using configuration files: ./docker-compose.yml,./docker-compose.override.yml
docker.auth.auth.load_config: Trying /root/.docker/config.json
docker.auth.auth.load_config: File doesn't exist
docker.auth.auth.load_config: Trying /root/.dockercfg
docker.auth.auth.load_config: File doesn't exist - returning empty config
compose.cli.command.get_client: Compose version 1.5.1
compose.cli.command.get_client: Docker base_url: http+docker://localunixsocket
compose.cli.command.get_client: Docker version: KernelVersion=3.19.0-37-generic, Os=linux, BuildTime=Fri Nov 20 13:12:04 UTC 2015, ApiVersion=1.21, Version=1.9.1, GitCommit=a34a1d5, Arch=amd64, GoVersion=go1.4.2
compose.service.pull: Pulling solr (dockerregistry.mycompany.de:6597/mycompany/imagename:latest)...
compose.cli.verbose_proxy.proxy_callable: docker pull <- ('dockerregistry.mycompany.de:6597/mycompany/imagename', tag=u'latest', stream=True)
docker.api.image.pull: Looking for auth config
docker.api.image.pull: No auth config in memory - loading from filesystem
docker.auth.auth.load_config: Trying /root/.docker/config.json
docker.auth.auth.load_config: File doesn't exist
docker.auth.auth.load_config: Trying /root/.dockercfg
docker.auth.auth.load_config: File doesn't exist - returning empty config
docker.auth.auth.resolve_authconfig: Looking for auth entry for 'dockerregistry.mycompany.de:6597'
docker.auth.auth.resolve_authconfig: No entry found
docker.api.image.pull: No auth config found
compose.cli.verbose_proxy.proxy_callable: docker pull -> <generator object _stream_helper at 0x7f112eb363c0>
Pulling repository dockerregistry.mycompany.de:6597/mycompany/imagename
ERROR: compose.cli.main.main: Error: image mycompany/imagename:latest not found
aanand commented 8 years ago

@gnarea This bit looks relevant:

docker.auth.auth.load_config: Trying /home/PROJECT_NAME/.dockercfg
docker.auth.auth.load_config: Attempting to parse as JSON
docker.auth.auth.load_config: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128)
docker.auth.auth.load_config: Attempting to parse legacy auth file format
docker.auth.auth.load_config: list index out of range
docker.auth.auth.load_config: All parsing attempts failed - returning empty config

Perhaps there's non-ASCII in your .dockercfg and we're not processing it correctly.

aanand commented 8 years ago

@sascha-egerer In your case it looks like we can't find a config file at all:

docker.auth.auth.load_config: Trying /root/.docker/config.json
docker.auth.auth.load_config: File doesn't exist
docker.auth.auth.load_config: Trying /root/.dockercfg
docker.auth.auth.load_config: File doesn't exist - returning empty config

Do you know the path to your config file?

sascha-egerer commented 8 years ago

@aanand ah fu**!

I know what the problem is... We have that problem on our CI Server. The admin of the ci server decided to run docker-compose as an docker-container but does not mount the config into the container. Thats why docker commands work but docker-compose do not. Thank you for the hint!

But the error message is a little bit misleading. Some like "could not authenticate against dockerregistry.mycompany.de:6597" would make way more sense then saying mycompany/imagename:latest

aanand commented 8 years ago

The "not found" error is a little out of our control - for security/privacy reasons, private repos return a 404 when authentication fails (GitHub does the same thing).

Still, we could feasibly print some debug information about where we looked for configuration if a pull fails for any reason.

sascha-egerer commented 8 years ago

@aanand Ah yes i see. But looks like this is not really consistent. A request to the API gives an 401 wich is what should be checked. And this is also documented here https://docs.docker.com/registry/spec/api/#api-version-check

$ curl -v https://dockerregistry.ffo.bar:443/v2/
...
< HTTP/1.1 401 Unauthorized
< Content-Type: application/json; charset=utf-8
< Docker-Distribution-Api-Version: registry/2.0
< Www-Authenticate: Basic realm="basic-realm"
< X-Content-Type-Options: nosniff
< Date: Fri, 18 Dec 2015 14:39:02 GMT
< Content-Length: 87
...

So an idea could be to check against that before pulling or after an 404 has been happened. What do you think about that?

gnarea commented 8 years ago

@aanand: I deleted .dockercfg and did "docker login" again, and now "docker-compose pull" works. Weird thing is that "docker pull FOO" did work. Thanks anyway!

aanand commented 8 years ago

@gnarea It's also possible that you had a legacy .dockercfg (how long ago did you last docker login?) and our parsing code for the legacy format is buggy.

gnarea commented 8 years ago

@aanand, that must be it, that file was quite old. Cheers!

ThomasEg commented 8 years ago

I just had the same issue with our private repositories on hub.docker.com...Whenever i tried to do a "compose-docker up -d" it failed with "Image not found" on all the private images(even though i tried a "docker login" everywhere(!))...

Docker version in my test setup(including swarm):

C:\>docker version
Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 21:49:11 2016
 OS/Arch:      windows/amd64

Server:
 Version:      swarm/1.1.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   7e9c6bd
 Built:        Wed Mar  2 00:15:12 UTC 2016
 OS/Arch:      linux/amd64

and compose:

C:\>docker-compose version
docker-compose version 1.6.2, build e80fc83
docker-py version: 1.7.2
CPython version: 2.7.11
OpenSSL version: OpenSSL 1.0.2d 9 Jul 2015

Solved it by doing a "docker pull [privateImage]" untill i got all the images pulled down. Then "docker-compose up" ran just fine! So, if you got the same "Image not found" when trying to compose using private images, just pull them first with Docker, then everything works...

A little weird that docker-compose is unable to pull the private images when docker itself have no problem doing so...anyways, the problem persists in the Docker versions above if anyone is interested...

sascha-egerer commented 8 years ago

@ThomasJoergensen is your docker-compose a local binary or is it a docker container itself? In most cases the reason for you problem is that the authentication credentials are missing.

Please post the output of docker-compose --verbose pull yourServiceName

igrcic commented 8 years ago

Hello,

is docker-compose supposed to work with insecure/self-signed registries (https://docs.docker.com/registry/insecure) ?

Using docker pull I get:

$ docker pull myserver:5000/my_image
Using default tag: latest
latest: Pulling from my_image
Digest: sha256:120a0f392551b447d....
Status: Image is up to date for myserver:5000/my_image:latest

Using docker-compose pull:

$ docker-compose pull myserver:5000/my_image --verbose
ERROR: No such service: myserver:5000/my_image:latest

docker-compose --version docker-compose version 1.7.1, build 0a9ab35

Thank you

aanand commented 8 years ago

@igrcic That error has nothing to do with the registry being insecure or self-signed. docker-compose pull expects a service name. If you have a service defined in your docker-compose.yml like this:

version: "2"
services:
  foo:
    image: myserver:5000/my_image

then docker-compose pull foo will pull the image.

Incidentally, the --verbose flag should come directly after docker-compose - sorry that it can't go at the end, I know it's annoying!

igrcic commented 8 years ago

Thnx @aanand,

I had replaced one underscore with the hyphen in my container name :P Everything is working as expected!

First i tried with --verbose after the pull, but it complained. It doesn't complain at the end (but obviously it doesn't work either :D )

Merci, Ivan

aanand commented 8 years ago

How does it complain? If there's a bug with --verbose, it'd be good to track it down.

igrcic commented 8 years ago

As you said, --verbose has to come right after the docker-compose. In other combinations we get:

$ docker-compose pull test --verbose
ERROR: No such service: --verbose
$ docker-compose pull --verbose test                                                                                                                 
Pulls images for services.

Usage: pull [options] [SERVICE...]

Options:
    --ignore-pull-failures  Pull what it can and ignores images with pull failures.
aanand commented 8 years ago

OK, that's expected (and also annoying, but difficult to fix). Thanks.

On Thu, 19 May 2016 at 16:02 igrcic notifications@github.com wrote:

As you said, --verbose has to come right after the docker-compose. In other combinations we get:

$ docker-compose pull test --verbose ERROR: No such service: --verbose

$ docker-compose pull --verbose test Pulls images for services.

Usage: pull [options] [SERVICE...]

Options: --ignore-pull-failures Pull what it can and ignores images with pull failures.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/docker/compose/issues/1622#issuecomment-220351993

hoto commented 8 years ago

This is a workaround for Windows 7.

We were running Docker Toolbox on Windows 7 with HOME system env setup to c:/home.
When we run docker-compose --verbose up it said it could not find auth file and could not pull image from private registry.
We copied c:/users/{user}/.docker/config.json to c:/home/.docker/config.json and it worked!

david-yu commented 8 years ago

I'm having the same issue even with the config file found, the following is my output from verbose mode on a compose pull:

vagrant@jenkins-node:~/docker-app-swarm-nginx$ docker-compose --verbose pull app 
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.auth.auth.load_config: Found 'auths' section
docker.auth.auth.parse_auth: Found entry (registry=u'172.28.128.4', username=u'admin')
compose.cli.command.get_client: docker-compose version 1.7.1, build 0a9ab35
docker-py version: 1.8.1
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
compose.cli.command.get_client: Docker base_url: https://172.28.128.3:443
compose.cli.command.get_client: Docker version: KernelVersion=3.19.0-61-generic, Os=linux, BuildTime=Fri May 27 17:25:03 UTC 2016, ApiVersion=1.22, Version=ucp/1.1.1, GitCommit=f28e3be, Arch=amd64, GoVersion=go1.6.2
compose.service.pull: Pulling app (172.28.128.4/docker-datacenter/docker-app-swarm-nginx:latest)...
compose.cli.verbose_proxy.proxy_callable: docker pull <- ('172.28.128.4/docker-datacenter/docker-app-swarm-nginx', tag=u'latest', stream=True)
docker.api.image.pull: Looking for auth config
docker.auth.auth.resolve_authconfig: Looking for auth entry for '172.28.128.4'
docker.auth.auth.resolve_authconfig: Found '172.28.128.4'
docker.api.image.pull: Found auth config
compose.cli.verbose_proxy.proxy_callable: docker pull -> <generator object _stream_helper at 0x7f541cf108c0>
jenkins-node: Pulling 172.28.128.4/docker-datacenter/docker-app-swarm-nginx:latest... : Error: image docker-datacenter/docker-app-swarm-nginx not found
dtr-node: Pulling 172.28.128.4/docker-datacenter/docker-app-swarm-nginx:latest... : Error: image docker-datacenter/docker-app-swarm-nginx not found
ucp-node: Pulling 172.28.128.4/docker-datacenter/docker-app-swarm-nginx:latest... : Error response from daemon: Get https://172.28.128.4/v1/_ping: x509: ceapp-node2: Pulling 172.28.128.4/docker-datacenter/docker-app-swarm-nginx:latest... : Error: image docker-datacenter/docker-app-swarm-nginx not found
app-node1: Pulling 172.28.128.4/docker-datacenter/docker-app-swarm-nginx:latest... : Error: image docker-datacenter/docker-app-swarm-nginx not found
lb-node: Pulling 172.28.128.4/docker-datacenter/docker-app-swarm-nginx:latest... : Error: image docker-datacenter/docker-app-swarm-nginx not found
app-node3: Pulling 172.28.128.4/docker-datacenter/docker-app-swarm-nginx:latest... : Error: image docker-datacenter/docker-app-swarm-nginx not found
ERROR: compose.cli.main.main: Error: image docker-datacenter/docker-app-swarm-nginx not found
WadeWaldron commented 8 years ago

I also encountered this issue. We were using docker login and not specifying a server. Using --verbose when doing the docker-compose pull or docker-compose up revealed that it was looking for a registry entry for docker.io but was unable to find one. I changed to docker login docker.io and everything started working.

Delved deeper and noticed that when I did a docker login and didn't specify a server, it would put an entry such as:

"": {
    "auth": "token"
}

However if I specify docker.io then I get:

"docker.io": {
    "auth": "token"
}

The interesting thing is that for doing a docker pull it seems unnecessary to supply the server name. It is only when doing a docker-compose pull that it matters.

aanand commented 8 years ago

@WadeWaldron OK, that speaks to an inconsistency in name matching - presumably the CLI has special logic for matching the empty string against docker.io.

Could you open an issue on https://github.com/docker/docker-py, please?

tnymlr commented 8 years ago

Hello! I experiencing same problem with stable version of "docker for windows". It seems that docker-compose unable to read credentials from windows credentials manager.

docker-compose:

~\P\D\devbox [master] > dc --version
docker-compose version 1.8.0, build d988a55

docker:

~\P\D\devbox [master] > d --version
Docker version 1.12.0, build 8eab29e

Login is successful:

~\P\D\devbox [master] > d login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username (dikeert):
Password:
Login Succeeded

Credentials stored using windows credentials store:

~\P\D\devbox [master] > cat C:\Users\anter\.docker\config.json
{
        "auths": {
                "https://index.docker.io/v1/": {}
        },
        "credsStore": "wincred"
}

docker-compose fails to download image:

~\P\D\devbox [master] > dc --verbose pull
compose.config.config.find: Using configuration files: .\docker-compose.yml
docker.auth.auth.load_config: Found 'auths' section
docker.auth.auth.parse_auth: Auth data for {0} is absent. Client might be using a credentials store instead.
docker.auth.auth.load_config: Found 'credsStore' section
compose.cli.command.get_client: docker-compose version 1.8.0, build d988a55
docker-py version: 1.9.0
CPython version: 2.7.11
OpenSSL version: OpenSSL 1.0.2d 9 Jul 2015
compose.cli.command.get_client: Docker base_url: http://127.0.0.1:2375
compose.cli.command.get_client: Docker version: KernelVersion=4.4.15-moby, Os=linux, BuildTime=2016-07-28T21:15:28.963402499+00:00, ApiVersion=1.24, Version=1.12.0, GitCommit=8eab29e, Arch=amd64, GoVersion=go1.6.3
compose.service.pull: Pulling devbox (dikeert/devbox:latest)...
compose.cli.verbose_proxy.proxy_callable: docker pull <- ('dikeert/devbox', tag=u'latest', stream=True)
docker.api.image.pull: Looking for auth config
docker.auth.auth.resolve_authconfig: Looking for auth entry for 'docker.io'
docker.auth.auth.resolve_authconfig: No entry found
docker.api.image.pull: No auth config found
compose.cli.verbose_proxy.proxy_callable: docker pull -> <generator object _stream_helper at 0x0000000003E3FEA0>
Pulling repository docker.io/dikeert/devbox
ERROR: compose.cli.main.main: Error: image dikeert/devbox:latest not found

Docker itself is able to pull image:

~\P\D\devbox [master] > docker pull dikeert/devbox
Using default tag: latest
latest: Pulling from dikeert/devbox
43db9dbdcb30: Already exists
2dc64e8f8d4f: Already exists
670a583e1b50: Already exists
183b0bfcd10e: Already exists
19c2e5ad56fb: Already exists
b19cc0948515: Already exists
f7d7f37f0242: Already exists
6618d8bf5d2f: Already exists
c274422cae73: Pull complete
5d421ef5bc6c: Pull complete
5f316f01bb10: Pull complete
7e18ba0554b5: Pull complete
660c68a22ca2: Pull complete
d1e01e92a984: Pull complete
fecb6bb94824: Pull complete
6652fee762b1: Pull complete
96a94be379d8: Pull complete
a52af9503e24: Pull complete
52af265303fe: Pull complete
dd2c736efe1e: Pull complete
caa058d3e790: Pull complete
d37994601e86: Pull complete
Digest: sha256:5e1b4c2b719d0508c45dac26e4cc690e735f13633ad253132a62cf3174cdfa44
Status: Downloaded newer image for dikeert/devbox:latest
creativeux commented 8 years ago

Also seeing this issue trying to set DTR up to manage our images. No problem pulling using docker pull, but docker-compose pull fails. Successful docker login and appropriate certificate installation into the local docker toolbox (have not been successful getting certificates installed using Docker for Mac).

UPDATE: Removed incorrect log trace from original post; I had pasted incorrect syntax with an unrelated error. See below for correct output.

docker-compose.yml:

version: '2'

networks:
  core:
  common:
  metrics:
  neo:

services:
  ##
  # Ticketing Service
  ##
  ticketing:
    image: dtr.gcihub.com/dig/ticketing-service:latest
    ports:
      - "8087:8080"
      - "9087:9009"
    networks:
      - neo
      - common
      - core
      - metrics
    mem_limit: 768m
    memswap_limit: 896m
    environment:
      - JAVA_OPTS=-server -Xms192m -Xmx192m
      - METRICS_DESTINATION=INFLUXDB
% docker-compose --verbose pull ticketing
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.auth.auth.load_config: Found 'auths' section
docker.auth.auth.parse_auth: Found entry (registry=u'art.gcihub.com', username=u'astone')
docker.auth.auth.parse_auth: Found entry (registry=u'dtr.gcihub.com', username=u'astone')
compose.cli.command.get_client: docker-compose version 1.8.0, build f3628c7
docker-py version: 1.9.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.2h  3 May 2016
compose.cli.command.get_client: Docker base_url: https://192.168.99.100:2376
compose.cli.command.get_client: Docker version: KernelVersion=4.4.16-boot2docker, Os=linux, BuildTime=2016-07-28T23:54:00.654661917+00:00, ApiVersion=1.24, Version=1.12.0, GitCommit=8eab29e, Arch=amd64, GoVersion=go1.6.3
compose.service.pull: Pulling ticketing (dtr.gcihub.com/dig/ticketing-service:latest)...
compose.cli.verbose_proxy.proxy_callable: docker pull <- (u'dtr.gcihub.com/dig/ticketing-service', tag=u'latest', stream=True)
docker.api.image.pull: Looking for auth config
docker.auth.auth.resolve_authconfig: Looking for auth entry for u'dtr.gcihub.com'
docker.auth.auth.resolve_authconfig: Found u'dtr.gcihub.com'
docker.api.image.pull: Found auth config
compose.cli.verbose_proxy.proxy_callable: docker pull -> <generator object _stream_helper at 0x1039f0cd0>
Pulling repository dtr.gcihub.com/dig/ticketing-service
ERROR: compose.cli.main.main: Error: image dig/ticketing-service:latest not found
% docker pull dtr.gcihub.com/dig/ticketing-service
Using default tag: latest
latest: Pulling from dig/ticketing-service
e110a4a17941: Pull complete 
1451fd43a838: Pull complete 
8b105f91d2aa: Pull complete 
6c1b097d7a34: Pull complete 
24a2b753c32f: Pull complete 
0969aee34eb7: Pull complete 
Digest: sha256:c1197990997683242058879b68b095d1e90562378e8d9f102485e76ead697fc8
Status: Downloaded newer image for dtr.gcihub.com/dig/ticketing-service:latest
ekristen commented 8 years ago

I'm seeing this problem with docker-compose 1.8.0 and docker daemon 1.12.1 via swarm 1.2.5 -- docker-compose pull works, but docker-compose up does not if the image does not exist locally.

The root problem is that on docker-compose up, docker-compose does not send the authorization header to the registry so the registry auth server doesn't get it.

I have access to the logs of both the registry and the auth services, this is how I can confirm this.

dopry commented 8 years ago

I am experiencing the same problem... docker pull works fine, docker-compose pull fails.

PS C:\Users\Darrel\src\spry-group\Cykik> docker-compose --verbose pull
compose.config.config.find: Using configuration files: .\docker-compose.yml
docker.auth.auth.load_config: Found 'auths' section
docker.auth.auth.parse_auth: Auth data for {0} is absent. Client might be using a credentials store instead.
docker.auth.auth.load_config: Found 'credsStore' section
compose.cli.command.get_client: docker-compose version 1.8.0, build d988a55
docker-py version: 1.9.0
CPython version: 2.7.11
OpenSSL version: OpenSSL 1.0.2d 9 Jul 2015
compose.cli.command.get_client: Docker base_url: http://127.0.0.1:2375
compose.cli.command.get_client: Docker version: KernelVersion=4.4.19-moby, Os=linux, BuildTime=2016-08-18T17:32:24.504694950+00:00, ApiVersion=1.24, Version=1.12.1, GitCommit=23cf638, Arch=amd64, Experimental=True, GoVersion=go1.6.3
compose.service.pull: Pulling api (sprygroup/cykik-api:latest)...
compose.cli.verbose_proxy.proxy_callable: docker pull <- ('sprygroup/cykik-api', tag=u'latest', stream=True)
docker.api.image.pull: Looking for auth config
docker.auth.auth.resolve_authconfig: Looking for auth entry for 'docker.io'
docker.auth.auth.resolve_authconfig: No entry found
docker.api.image.pull: No auth config found
compose.cli.verbose_proxy.proxy_callable: docker pull -> <generator object _stream_helper at 0x00000000040A6708>
Pulling repository docker.io/sprygroup/cykik-api
ERROR: compose.cli.main.main: Error: image sprygroup/cykik-api:latest not found
shin- commented 8 years ago

Hey folks,

I just wanted to update here to say we've done a pass on some of the recent auth issues in docker-py, notably added support for credential stores if you're using one of those, and support for identity tokens (if you are using Compose with DTR or UCP, this probably concerns you). Those fixes will be in the next Compose release, tentatively 1.8.1.

Related issues:

shin- commented 8 years ago

With 1.8.1 being released last week, I am now going to close this issue. If you're still having trouble with authentication, please create a new issue describing the problem!