Help me understand the --preserve-digests and --all options for copy command. How do I preserve the image digest during copying?

[evsasha]

Help me understand the --preserve-digests and --all options for copy command. How do I preserve the image digest during copying?

I read the documentation, applied the --preserve-digests and --all options, but the digest still changes.

$ skopeo --version
skopeo version 1.13.3

# inspect
$ skopeo inspect docker://registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20231011-8b53cabe0 | grep -i digest -m 1
    "Digest": "sha256:a7943503b45d552785aa3b5e457f169a5661fb94d82b8a3373bcd9ebaf9aac80",

# copy with no flags
$ skopeo copy docker://registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20231011-8b53cabe0 dir:/tmp
$ cat /tmp/manifest.json | grep -i digest -m 1
    "digest": "sha256:1ebff0f9671bc015dc340b12c5bf6f3dbda7d0a8b5332bd095f21bd52e1b30fb",

# copy with --preserve-digests flag
$ skopeo copy --preserve-digests  docker://registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20231011-8b53cabe0 dir:/tmp
$ cat /tmp/manifest.json | grep -i digest -m 1
    "digest": "sha256:1ebff0f9671bc015dc340b12c5bf6f3dbda7d0a8b5332bd095f21bd52e1b30fb",

# copy with --all --preserve-digests flags
$ skopeo copy --all --preserve-digests  docker://registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20231011-8b53cabe0 dir:/tmp
$ cat /tmp/manifest.json | grep -i digest
      "digest": "sha256:29318c6957228dc10feb67fed5b91bdd8a9e3279e5b29c5965b9bd31a01ee385",
      "digest": "sha256:bd8698283ba2f8d83dd71b6b75f2f7f8585a87049f61da2a68bbc25bd8caf588",
      "digest": "sha256:67202a0258c6f81d073f265f449a732c89cc1112a8e80ea27317294df6dce2b5",
      "digest": "sha256:5b4f4d4c65f84f9a6d6c36dee0fdfa51c6632a8cd6f47dcd0b1c06ba33951109",

[mtrmac]

Thanks for reaching out.

cat /tmp/manifest.json | grep -I digest is just not a way to obtain the digest of the result of a copy. That’s not how it works.

You already know one way to compute the digest:

% skopeo inspect dir:t |grep Digest -m 1
    "Digest": "sha256:a7943503b45d552785aa3b5e457f169a5661fb94d82b8a3373bcd9ebaf9aac80",

matches, when --all (and --preserve-digests, which is not strictly necessary for dir: destinations without other options, but is useful as a safeguard) is used.

[evsasha]

Thank you for the comment. I deliberately hid the full output of the inspect command and the contents of the manifest.json files to focus the user's attention solely on my question - the digests are different. You can be confident in the accuracy of the data provided above.

I specifically used a public image in the example that is accessible to everyone to demonstrate the issue. Anyone can try to copy the image using skopeo and see that the digest has changed.

I provided all the optional keys for the copy command, as similar issues often suggest trying various combinations. In my case, none of the combinations helped.

I simply want to understand why the image digest might change and how to discern this before copying the image.

Interestingly, this image will retain its digest when using 'docker pull,' whereas the digest will change when pushing it to a remote repository with 'docker push.'

[mtrmac]

The reproduction steps are sufficient, and I have followed them. The digest didn’t change with --all --preserve-digests.

You are asking a relevant question but using incorrect tools to determine the answer. skopeo inspect is a possible tool. Sure, skopeo inspect | grep, if you want to (although I’d recommend something more reliable like jq).

grep manifest isn’t relevant at all (because the digest is a hash of the manifest, in many but not all cases).

[evsasha]

@mtrmac Thank you. You are right. Indeed, I used the wrong method to compare the digests. Below is the correct example.

$ skopeo inspect docker://registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20231011-8b53cabe0 | grep Digest -m 1
    "Digest": "sha256:a7943503b45d552785aa3b5e457f169a5661fb94d82b8a3373bcd9ebaf9aac80",

$ skopeo copy --all --preserve-digests  docker://registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20231011-8b53cabe0 dir:/tmp
$ skopeo inspect dir:/tmp | grep Digest -m 1
    "Digest": "sha256:a7943503b45d552785aa3b5e457f169a5661fb94d82b8a3373bcd9ebaf9aac80",

The task can be closed.

[mtrmac]

Thanks for the confirmation.