search

fabric8io / docker-maven-plugin

Maven plugin for running and creating Docker images
https://dmp.fabric8.io
Apache License 2.0
1.88k stars 643 forks source link

Can't match GCR auth config in ~/.docker/config.json #1034

Open seboudry opened 6 years ago

seboudry commented 6 years ago

Description

DMP can't match an existing auth section in ~/.docker/config.json to GCR.

Info

Reproduce bug

Log to GCR:

docker login eu.gcr.io

Try to use the auth section in ~/.docker/config.json:

fabric8:push -Ddocker.push.registry=eu.gcr.io/awesome

The auth configuration does not match "eu.gcr.io" != "eu.gcr.io/awesome"

[INFO] --- fabric8-maven-plugin:3.5.38:push (default-cli) @ qs-framework-fabric8-python-samples-echo ---
[DEBUG] F8> AuthConfig: no credentials found
rhuss commented 6 years ago

"eu.gcr.io/awesome" doesn't look like a registry, its the registry + repository name. Both concepts are clearly separate within d-m-p, so my question to you is why is "awesome" not part of the image name (as specified within the image configuration), using only "eu.gcr.io" as registry name ?

seboudry commented 6 years ago

For my use case, repository name can't be part of image name. For now (with f-m-p), image names consist of last part of groupId and artifactId. So, repository (in this case GCP project) don't have place there. Can have multiple projects in same repository (like in Nexus). Moreover, I want to be free for where I store my Docker images. Same image can be in multiple registries (CI and production for example). I mention the target registry only at deployment phase.

PS : I use both "repository" and "registry" words on this answer. They maybe not have same signification.

rhuss commented 6 years ago

So in the context of d-m-p a "registry" is alwasy a pure registry. It's quay.io, docker.io, or eu.gcr.io. The "repository" is the user name within this registry. It's part of the image name. Please don't confuse them.

So, when pushing to different registry with the same user name, used docker.registry (or the equivalent XML config). If you want to have different user names for you images use something like this in the configuration:

<images>
  <image>
    <name>${image.user}/myimage</name>
  </image>
</images>

and then user -Dimage.user=itsme on the commandline or something like this in the properties section.

But dont add the user part to the registry name when using -Ddocker.push.registry. That's not a registry anymore then.

seboudry commented 6 years ago

To be clear and use f-m-p dialect, in the case of GCR:

When you authenticate on GCR, you use the full registry name (including project ID). And then, with docker command, the full image name (with repository) is used.

So, when using f-m-p with GCR we end up with this kind of URI: eu.gcr.io/my-gcp-project/my-group/my-app.

In our use case, when we move image across GCP project, we retag by changing the project ID in registry name and keeping the full image name.

Do you see the point of this issue? It's a bit confusing because docker login command take only eu.gcr.io as argument, because we run gcloud auth configure-docker before that use project ID.

rhuss commented 6 years ago

@seboudry my intepretation of cloud.google.com/container-registry/docs/overview#registry_name is indeed different:

That similar to

When you do an authentication against docker.io you also do it per user/project.

For your example I consider my-gcp-project/my-group/my-app being the image name, eu.gcr.io is the registry (a registry name is always a hostname). If you move your image across projects in the same registry, you should parameterise that part in the image name as suggested above with a Maven property.

Might be also that you confuse that 4-level names with this deprecated feature from GCR:

Domain-scoped projects

If your project is scoped to your domain, then the project ID includes the domain and a colon (:). Because of how Docker treats colons, you need to identify the images in these types of projects by using the following format:

[HOSTNAME]/[DOMAIN]/[PROJECT]/[IMAGE]

For example, the project named example.com:my-project could have the following image:

gcr.io/example.com/my-project/image-name

Note: Scoping projects to a domain is a legacy feature. You cannot create new domain-scoped projects.

(from https://cloud.google.com/container-registry/docs/overview#registry_name)

So, clearly in your example "my-gcp-project" and "my-group" belong together as for me this translates to a 'real' project id/username part "my-gcp-project:my-group" (considering you 'my-gcp-project' to be a domain).

A registry is definitely everything up to the first slash, when this first part contains either a . or a : --> https://stackoverflow.com/questions/37861791/how-are-docker-image-names-parsed#37867949 (the hostname there is the registry) So a registry can never contain a slash.

seboudry commented 6 years ago

Thanks for your interpretation. We'll stick with that.

Nope, we don't use domain-scoped projects (indeed deprecated). In GCR, you can have some subfolders inside your registry/repository (ie. the my-group between my-gcp-project and image-name). I've search for documentation, but can't find it ^^

For example, I can do:

docker pull fabric8/maven-builder
docker tag fabric8/maven-builder eu.gcr.io/my-gcp-project/fabric8/maven-builder
gcloud auth configure-docker
docker push eu.gcr.io/my-gcp-project/fabric8/maven-builder

But that's not the matter of this issue and behaviour of f-m-p using groupId don't bother us with GCR subfolder.

Maybe some tests using the "user" as GCP project ID would figure out a solution (using a JSON key as password or not). Unfortunately, I don't have some time for now. Here's the different kind of GCR authentication : https://cloud.google.com/container-registry/docs/advanced-authentication

rhuss commented 6 years ago

@seboudry not sure if I understand correctly, but why isn't it possible to use a registry "eu.gcr.io" and attach "awesome/" as prefix to the image name as configured in the image configuration ?

seboudry commented 6 years ago

That's because I don't want gcp-project-id to be part of my image name (consisted of group-id/artifact-id).

We use 2 GCP projects that holds GCR Docker registries:

This way, we can garantee that only images in GCP production project (after validation process) can be run in production. During deployment we prefix image name with corresponding GCR registry name so that Kubernetes can reach the image.

We keep group-id in image name because it allows separation on different company teams. And because GCR allows it.

We kept this mecanism because we used before the workaround explained here. For now, we use docker tag then gcloud auth configure-docker and docker push.

I undestand now that's a bad practice following FMP convention. So I believe that we can close this issue on this consideration, to avoid misconfiguration and code polution on FMP. Do you agree ? Not a problem for us as we use plain docker commands instead of FMP for the push phase.

presidentio commented 3 years ago

I have a similar issue, but with index.docker.io. Steps to Reproduce: 1) Login to docker hub using docker login 2) Make sure you have https://index.docker.io/v1/ entry in ~/.docker/config.json file 3) Create docker file that starts with FROM index.docker.io/adoptopenjdk/openjdk11:jre-11.0.6_10-alpine 4) Run maven command to build this image using maven-docker-plugin with -Ddocker.verbose=true -Dorg.slf4j.simpleLogger.log.io.fabric8.maven.docker=debug

Actual result: [DEBUG] DOCKER> AuthConfig: no credentials found Expected result: https://index.docker.io/v1/ authentication used to pull the image

As I understand the root cause of my issue is that docker-maven-plugin use exact string comparison of docker registry from image name and registry name from ~/.docker/config.json file. In my case: "https://index.docker.io/v1/".equals("index.docker.io") is false