Push Docker Attestation: Support credential helpers

[hfhbd]

I use Google Cloud Artifactory Registry and the google auth action. This action adds a credential helper to the docker config file, but I get Error: No credentials found for registry europe-west4-docker.pkg.dev:

Run gcloud auth configure-docker europe-west4-docker.pkg.dev

Adding credentials for: europe-west4-docker.pkg.dev
After update, the following will be written to your Docker config file located 
at [/home/runner/.docker/config.json]:
 {
  "credHelpers": {
    "europe-west4-docker.pkg.dev": "gcloud"
  }
}

Error:

Run actions/attest-build-provenance@v1
Run actions/attest-build-provenance/predicate@db1dde0f270afe12073070ac7aa802958ae3ec04
Run actions/attest@12c083815ed46d5d78222e3824f4a26c42c234d3
Attestation created for europe-west4-docker.pkg.dev/***/composetodo-repo/todo@sha256:9c9e5d7262be6bc0ead3bbf6b01dbfa8557d0a0e6108f0a285d3f39efde1d1fa
Attestation signed using certificate from Public Good Sigstore instance
Attestation signature uploaded to Rekor transparency log
[https://search.sigstore.dev?logIndex=94477234](https://search.sigstore.dev/?logIndex=94477234)
Attestation uploaded to repository
https://github.com/hfhbd/ComposeTodo/attestations/859947
Error: Error: No credentials found for registry europe-west4-docker.pkg.dev

Job log: https://github.com/hfhbd/ComposeTodo/actions/runs/9118458625/job/25071641672

[bdehamer]

At the moment, the push-to-registry option does not work with credential helpers and relies on the token being present in the Docker config.

I'm not sure if this is reasonable for your use case, but if you pair the google-github-actions/auth action with the docker/login-action action it should work (see example here authenticating against Google Container Registry)

[hfhbd]

Thanks, using the access token did work.

[hfhbd]

Okay, it still does not work, but this time with another error:

Error fetching https://europe-west4-docker.pkg.dev/v2/***/composetodo-repo/todo/manifests/sha256-907b3557d0ac0acfa8884039a43bec77a1cc6c00d3cd0b03612795c78cae2084 - expected 201, received 400

logs: https://github.com/hfhbd/ComposeTodo/actions/runs/9138975997/job/25130765410 workflow: https://github.com/hfhbd/ComposeTodo/blob/v0.0.46/.github/workflows/CD.yml

Can you try your demo with a non-latest tag too? (Just to test?)

[bdehamer]

@hfhbd I can't quite explain why that particular invocation failed (though it looks like subsequent builds on that repo succeeded without issue).

The failure you reported did expose a bug on our side (note how the output from actions/attest-build-provenance indicates that the push succeeded, but also raises an error). We've got a fix for this which will go out in the next release.

[cpanato]

Got a similar issue when pushing to Docker hub and having both docker login steps in place (for docker.io and ghcr.io)

https://github.com/cpanato/dex/actions/runs/9252670770/job/25450736306

[hfhbd]

Yeah, I still get an 400 when using the new 1.2.0 version:

https://github.com/hfhbd/ComposeTodo/actions/runs/9434453487/job/25986646005

[bdehamer]

@hfhbd do you happen to have immutable tags enabled on your repository? This could explain the error that you are seeing.

When the attestation is pushed to the repository, an index is created which lists all of the artifacts which are associated with your image:

{
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:798aff77e7d52108b04e84a99c29e9084f1f39f71e30c53e6936544ad32ef0fe",
      "size": 798,
      "artifactType": "application/vnd.dev.sigstore.bundle.v0.3+json",
      "annotations": {
        "org.opencontainers.image.created": "2024-06-12T15:44:52.466Z",
        "dev.sigstore.bundle.content": "dsse-envelope",
        "dev.sigstore.bundle.predicateType": "application/json"
      }
    }
  ]
}

This index is tagged with a name which references the digest of the associated image. In the event that multiple artifacts are attached to the same image, this index may get updated and re-tagged . . . which would fail if you had the immutable tags option enabled on your repo.

[hfhbd]

@bdehamer Yes indeed, I enabled immutable tags. Is this option not supported with attestations?

[bdehamer]

Is this option not supported with attestations?

Under some conditions it may work, but if you run into errors like the one you're seeing the best option is to disable the immutable tags option.

Some container registries have support for something called the referrers API which completely solves this issue. With the referrers API, the registry itself will keep track of any referring artifacts associated with your image and automatically maintain an index of those artifacts.

For registries which do NOT support the referrers API, the recommended approach is to manually maintain the artifact index and tag it with a reference to the image's digest (sha256-<image-digest>). Unfortunately, Google Artifact Registry does not support the referrers API so we're using the tag fallback.

If all of your builds result in distinct images (with distinct digests) AND you're not creating multiple attestations for the same image you may find that things work just fine even if you have immutable tags enabled. If the artifact index tag is never moved there should be no problem.

The immutable tags feature is a nice security feature, and disabling it isn't ideal -- however, you can get the same guarantees by verifying your attestations:

gh attestation verify oci://europe-west4-docker.pkg.dev/composetodo-repo/todo:0.0.48.200 -R hfhbd/ComposeTodo

This command would report an error if the 0.0.48.200 tag were moved to point to a new image after the attestation was created.

[hfhbd]

@bdehamer Thank you for your detailed analysis, explanation and the workaround, I really appreciate it!

I also created a feature request at Google to implement the referrer api: https://issuetracker.google.com/issues/346827093 I did link your comments in the request, but feel free to comment if you want to.

I guess there are no todos on your side, so I will close this issue.