search

GoogleContainerTools / kaniko

Build Container Images In Kubernetes
Apache License 2.0
14.79k stars 1.44k forks source link

docker-credential-gcr: Workload Identity Federation #1889

Closed hermanbanken closed 2 years ago

hermanbanken commented 2 years ago

The library kaniko is using for GCR authentication is updated to support Workload Identity Federation (type = external_account). See issue https://github.com/GoogleCloudPlatform/docker-credential-gcr/issues/95 . Unfortunately Kaniko is using an older version so we have the following issue.

Actual behavior

error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "eu.gcr.io/$PROJECT/repo": resolving authorization for eu.gcr.io failed: error getting credentials - err: exit status 1, out: `docker-credential-gcr/helper: could not retrieve GCR's access token: google: error getting credentials using GOOGLE_APPLICATION_CREDENTIALS environment variable: unknown credential type: "external_account"`

Expected behavior Working pull/push/builds.

To Reproduce Steps to reproduce the behavior:

  1. Create a GitHub Action workflow.
  2. Use https://github.com/google-github-actions/auth to prepare authentication. This sets GOOGLE_APPLICATION_CREDENTIALS to a file with the format as ducmented in https://github.com/GoogleCloudPlatform/docker-credential-gcr/issues/95.
  3. Use Kaniko to build an image

Additional Information

Notes

I have a hard time figuring out if this should work: whether or not the change is in or out on GitHub-master and on GCR-latest.

This line must be present for it to work: https://cs.opensource.google/go/x/oauth2/+/master:google/google.go;l=97;drc=ba495a64dcb59dd28baeda5ded53504191dea54f it was implemented in this PR https://go-review.googlesource.com/c/oauth2/+/276312, merged on Tue Jan 12 19:45:47 2021.

Kaniko uses https://github.com/GoogleCloudPlatform/docker-credential-gcr/commits/4cdd60d0f2d8a69bc70933f4d7718f9c4e956ff8

RUN go install github.com/GoogleCloudPlatform/docker-credential-gcr@4cdd60d0f2d8a69bc70933f4d7718f9c4e956ff8

which is using golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f.

Weirdly enough, it is not working in latest. Where does one quickly find what is in latest on GCR?

Update: running docker-credential-gcr version inside the debug image reports Google Container Registry Docker credential helper 2.0.1.

Only version 2.1.0 supports external_account, which was added when they moved to Go Modules.

hermanbanken commented 2 years ago

Actually, version 4cdd60d0f2d8a69bc70933f4d7718f9c4e956ff8 is 2.1.0. So the main branch is including support already, it just needs to be released!

hermanbanken commented 2 years ago

Tested with Kaniko gcr.io/kaniko-project/executor:76624697df879f7c3e3348f22b8c986071af4835-debug which is the current main version:

WARN[0000] Failed to setup Google env authenticator, ignoring: google: error getting credentials using GOOGLE_APPLICATION_CREDENTIALS environment variable: unknown credential type: "external_account"

still happens.

Caused by https://github.com/GoogleContainerTools/kaniko/blob/986121553be04f5cd23f96e0d0443cd6212cfb25/pkg/creds/creds_linux.go#L44-L46, due to the vendoring of https://github.com/GoogleContainerTools/kaniko/blob/main/vendor/golang.org/x/oauth2/google/default.go which is pretty old.

imjasonh commented 2 years ago

That's calling through https://github.com/google/go-containerregistry/blob/19c2b639fab1/pkg/v1/google/auth.go#L50 (which is indeed pretty old!)

https://github.com/GoogleContainerTools/kaniko/pull/1885 bumps both go-containerregistry to main, and x/oauth2 to d3ed0bb246c8 which includes the external_account support.

hermanbanken commented 2 years ago

creds.json:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
  },
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/SERVICE_ACCOUNT_EMAIL:generateAccessToken"
}

Dockerfile:

FROM alpine

test.sh:

GOOGLE_APPLICATION_CREDENTIALS=/pwd/creds.json /kaniko/executor --destination=gcr.io/somewhere/overthe/rainbow --dockerfile=/pwd/Dockerfile -v=debug

Quick test:

# tag 1.6.0 / debug
docker run -it -v $PWD:/pwd --entrypoint sh gcr.io/kaniko-project/executor:a0c96b4edcaeee437243484f5880833518d06551-debug /pwd/test.sh -v=debug
# Output: 
# error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "gcr.io/somewhere/overthe/rainbow": resolving authorization for gcr.io failed: error getting credentials - err: exit status 1, out: `docker-credential-gcr/helper: could not retrieve GCR's access token: google: error getting credentials using
# GOOGLE_APPLICATION_CREDENTIALS environment variable: unknown credential type: "external_account"`

# 1.7.0
docker run -it -v $PWD:/pwd --entrypoint sh gcr.io/kaniko-project/executor:28a6bdce78a5f5cb09cc4cc650595721554eef6b-debug /pwd/test.sh -v=debug
# Output, as expected with fake credential file: 
# dial tcp 169.254.169.254:80: connect: connection refused

# main branch after PR 1890
docker run -it -v $PWD:/pwd --entrypoint sh gcr.io/kaniko-project/executor:6bbd8a331cbd361fe6dadcc2089adc4eb0e8ca08-debug /pwd/test.sh -v=debug
# Output, as expected with fake credential file: 
# dial tcp 169.254.169.254:80: connect: connection refused