Image doesnt include docker-credential-ecr-login for AWS credential helper

[olivermussell]

I’m running a Gitlab CI runner instance on Kubernetes (EKS) in AWS, and using an IRSA for authentication/permissions. manifest-tool is run in the pipeline once the multiple architecture images are created and pushed to the container registry (ECR). But when running manifest-tool to push the final manifest it was producing this error:

time=“2022-11-30T13:22:04Z” level=fatal msg=“Inspect of image \“$my_aws_account_id.dkr.ecr.us-east-1.amazonaws.com/$my_image_name:$my_tag_name-amd64\” failed with error: error getting credentials - err: exec: \“docker-credential-ecr-login\“: executable file not found in $PATH, out: ``”

Which appears to be fixed by installing the docker-credential-ecr-login package in the alpine container prior to running the manifest-tool binary. I'm unsure if its correct to be installing this package or if there is something else required for it to pick up the credentials correctly.

For context, here’s how its run in the gitlab-ci.yml:

multi_arch_manifest:
  stage: build_multi_arch_image
  variables:
    AWS_SDK_LOAD_CONFIG: “true”
    AWS_EC2_METADATA_DISABLED: “true”
  tags:
    - eks
    - amd64
  needs: [“build_image_amd64", “build_image_arm64”]
  image: mplatform/manifest-tool:alpine
  script:
  - echo ‘{“credsStore”:“ecr-login”}’ > /config.json
  - apk add docker-credential-ecr-login                       <---- This fixes this issue
  - manifest-tool --docker-cfg /config.json push from-args
    --platforms linux/amd64,linux/arm64
    --template “$my_aws_account_id.dkr.ecr.us-east-1.amazonaws.com/$my_image_name:$my_tag_name-ARCHVARIANT”
    --target “$my_aws_account_id.dkr.ecr.us-east-1.amazonaws.com/$my_image_name:multiarch”

[estesp]

Yes, since you are setting the credential store to ecr-login (in the script above, piped to config.json) your instance has to have the credential helper that provides that auth capability; installing it from the from the alpine package repo makes sense.

[olivermussell]

Ok thanks

If anyone else finds this issue, you would need to do a similar thing for Azure (docker-credential-acr-env) and GCP (docker-credential-gcr).

[aries1980]

While I understand supporting cloud providers with this image is stretch of a scope, yet I feel if you don't add support credential for helpers to the image like Kaniko does, you sentence the end-users to maintain their own image, hack around it with mounting from the host (if that's an option) or do a wasteful install of the credential helper on every single run.

I would recommend to reconsider adding it to the manifest-tool images.

[estesp]

Working on this via #216 now; forgot there had been a prior issue. Trying to decide if it should be a separate image or at least still have an image without the helpers as it increases the size dramatically:

manifest-tool              helpers    a9fb77bdea49    59 minutes ago       linux/arm64       44.9 MiB     21.7 MiB
mplatform/manifest-tool    alpine     4cdcd8344180    3 seconds ago        linux/arm64/v8    17.4 MiB     6.7 MiB
mplatform/manifest-tool    latest     ea6d7699ef23    3 weeks ago          linux/arm64/v8    9.7 MiB      3.6 MiB
mplatform/manifest-tool    v2.0.8     ea6d7699ef23    3 weeks ago          linux/arm64/v8    9.7 MiB      3.6 MiB