search

oras-project / oras

OCI registry client - managing content like artifacts, images, packages
https://oras.land
Apache License 2.0
1.52k stars 181 forks source link

fix: disable oras tag fallback to tag schema when tagging a referrer #1435

Closed njucjc closed 5 months ago

njucjc commented 5 months ago

What this PR does / why we need it:

Please check the following list:

qweeah commented 5 months ago

Hi @njucjc, thanks for contributing. Can you help explain why --distribution-spec flag need to be added to oras tag?

njucjc commented 5 months ago

Hi @njucjc, thanks for contributing. Can you help explain why --distribution-spec flag need to be added to oras tag?

@qweeah I want to tag a referrer, but now if not specify --distribution-spec v1.1-referrers-api, it will fallback to --distribution-spec v1.1-referrers-tag method. The result is that an unexpected addition of a sha256 subject tag in addition to the one I specified.

qweeah commented 5 months ago

I want to tag a referrer, but now if not specify --distribution-spec v1.1-referrers-api, it will fallback to --distribution-spec v1.1-referrers-tag method.

That's weird because --distribution-spec should only work for oras attach when adding a referrer. Can you kindly share the error debug log and which registry you are using?

njucjc commented 5 months ago

That's weird because --distribution-spec should only work for oras attach when adding a referrer. Can you kindly share the error debug log and which registry you are using?

@qweeah 

DEBU[0000] Request #0
> Request method: "GET"
> Request headers:
   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
   "User-Agent": "oras/1.2.0"
DEBU[0000] Response #0
< Response Status: "401 Unauthorized"
< Response headers:
   "Content-Length": "157"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Date": "Fri, 28 Jun 2024 14:31:27 GMT"
   "Content-Type": "application/json; charset=utf-8"
DEBU[0000] Request #1
> Request method: "GET"
> Request headers:
   "Authorization": "*****"
   "User-Agent": "oras/1.2.0"
DEBU[0000] Response #1
< Response Status: "200 OK"
< Response headers:
   "X-Xss-Protection": "1; mode=block"
   "Cache-Control": "no-cache, no-store, max-age=0, must-revalidate"
   "Pragma": "no-cache"
   "Timing-Allow-Origin": "*"
   "Server": "Tengine"
   "Vary": "Accept-Encoding"
   "X-Content-Type-Options": "nosniff"
   "Date": "Fri, 28 Jun 2024 14:31:27 GMT"
   "Strict-Transport-Security": "max-age=31536000"
   "Eagleeye-Traceid": "0a06dff517195850875481284ea292"
   "Content-Type": "application/json;charset=UTF-8"
   "Set-Cookie": "*****"
   "Expires": "0"
   "X-Frame-Options": "DENY"
DEBU[0000] Request #2
> Request URL: "https:/xxxx/manifests/sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb"
> Request method: "GET"
> Request headers:
   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
   "Authorization": "*****"
   "User-Agent": "oras/1.2.0"
DEBU[0000] Response #2
< Response Status: "200 OK"
< Response headers:
   "Date": "Fri, 28 Jun 2024 14:31:27 GMT"
   "Content-Type": "application/vnd.oci.image.index.v1+json"
   "Content-Length": "736"
   "Docker-Content-Digest": "sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Etag": "\"sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb\""
Tagging [registry] xxxxx/library/test-2@sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb
DEBU[0000] Request #3
> Request URL: "https://xxxxx/v2/library/test-2/manifests/test"
> Request method: "PUT"
> Request headers:
   "Content-Type": "application/vnd.oci.image.index.v1+json"
   "User-Agent": "oras/1.2.0"
DEBU[0000] Response #3
< Response Status: "401 Unauthorized"
< Response headers:
   "Content-Length": "230"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Www-Authenticate": "Bearer realm=\
   "Date": "Fri, 28 Jun 2024 14:31:27 GMT"
   "Content-Type": "application/json; charset=utf-8"
DEBU[0000] Request #4
> Request URL: 
> Request method: "GET"
> Request headers:
   "Authorization": "*****"
   "User-Agent": "oras/1.2.0"
DEBU[0001] Response #4
< Response Status: "200 OK"
< Response headers:
   "Date": "Fri, 28 Jun 2024 14:31:28 GMT"
   "X-Frame-Options": "DENY"
   "Strict-Transport-Security": "max-age=31536000"
   "Set-Cookie": "*****"
   "X-Xss-Protection": "1; mode=block"
   "Cache-Control": "no-cache, no-store, max-age=0, must-revalidate"
   "Pragma": "no-cache"
   "Expires": "0"
   "Eagleeye-Traceid": "0a06dff517195850879751310ea292"
   "Timing-Allow-Origin": "*"
   "Server": "Tengine"
   "Content-Type": "application/json;charset=UTF-8"
   "Vary": "Accept-Encoding"
   "X-Content-Type-Options": "nosniff"
DEBU[0001] Request #5
> Request URL: "https://xxxx/v2/library/test-2/manifests/test"
> Request method: "PUT"
> Request headers:
   "Content-Type": "application/vnd.oci.image.index.v1+json"
   "Authorization": "*****"
   "User-Agent": "oras/1.2.0"
DEBU[0001] Response #5
< Response Status: "201 Created"
< Response headers:
   "Date": "Fri, 28 Jun 2024 14:31:28 GMT"
   "Content-Length": "0"
   "Location": "https://xxxx/v2/library/test-2/manifests/sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb"
   "Docker-Content-Digest": "sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb"
   "Docker-Distribution-Api-Version": "registry/2.0"
DEBU[0001] Request #6
> Request URL: "https://xxxxxxx/v2/library/test-2/manifests/sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde"
> Request method: "GET"
> Request headers:
   "User-Agent": "oras/1.2.0"
   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
   "Authorization": "*****"
DEBU[0001] Response #6
< Response Status: "200 OK"
< Response headers:
   "Date": "Fri, 28 Jun 2024 14:31:28 GMT"
   "Content-Type": "application/vnd.oci.image.index.v1+json"
   "Content-Length": "302"
   "Docker-Content-Digest": "sha256:ce9946afdcd5298cc6da3940e2ebee9583e1c87a71d9ad2e5e0d2792f59626d5"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Etag": "\"sha256:ce9946afdcd5298cc6da3940e2ebee9583e1c87a71d9ad2e5e0d2792f59626d5\""
Tagged test

the registry (support referrers api as distribution-spec v1.1) appears two tags: test and sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde

By the way,oras cp subcommand will also use --from-distribution-spec and --to-distribution-spec flag to indicate that the remote registries support the Referrers API.

codecov[bot] commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 85.22%. Comparing base (ce758bc) to head (8bd4797).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1435 +/- ## ========================================== - Coverage 85.27% 85.22% -0.05% ========================================== Files 106 106 Lines 3796 3798 +2 ========================================== Hits 3237 3237 - Misses 334 336 +2 Partials 225 225 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

qweeah commented 5 months ago

the registry (support referrers api as distribution-spec v1.1) appears two tags: test and sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde

Just to confirm, so you are trying to avoid generating sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde and oras tag --distribution-spec v1.1-referrers-api is the desired behavior in your case, right?

shizhMSFT commented 5 months ago

Although tagging a referrer artifact manifest is discouraged, it is still doable. However, re-tagging is implemented as pushing the exact manifest again to the remote server. According to the distribution-spec, if the remote server does not support referrers API, it will go for the referrers tag schema case. However, it is not necessary since the manifest is already there, we are just adding a new tag not really pushing a new manifest.

In summary, tagging a referrers manifest should never go for the tag schema even if the server does not support referrers API since the referrer manifest is already there on the server.

njucjc commented 5 months ago

the registry (support referrers api as distribution-spec v1.1) appears two tags: test and sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde

Just to confirm, so you are trying to avoid generating sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde and oras tag --distribution-spec v1.1-referrers-api is the desired behavior in your case, right?

@qweeah Yes!

njucjc commented 5 months ago

Although tagging a referrer artifact manifest is discouraged, it is still doable. However, re-tagging is implemented as pushing the exact manifest again to the remote server. According to the distribution-spec, if the remote server does not support referrers API, it will go for the referrers tag schema case. However, it is not necessary since the manifest is already there, we are just adding a new tag not really pushing a new manifest.

In summary, tagging a referrers manifest should never go for the tag schema even if the server does not support referrers API since the referrer manifest is already there on the server.

@shizhMSFT @qweeah Is there any suggestions to fix it? The simplest way to disable the tag schema is set referrersAPI = true inside the registry client in oras tag subcommand.

qweeah commented 5 months ago

@shizhMSFT @qweeah Is there any suggestions to fix it? The simplest way to disable the tag schema is set referrersAPI = true inside the registry client in oras tag subcommand.

@njucjc Yes, we should disable tag schema without adding --distribution-spec to oras tag. If you are interested in fixing it, you may achieve that using https://pkg.go.dev/oras.land/oras-go/v2@v2.5.0/registry/remote#Repository.SetReferrersCapability.

qweeah commented 5 months ago

I have added two test specs, one of them ensures that tag schema is not used after tagging to an OCI v1.0 registry (which doesn't support referrers API). @njucjc With a proper fix in oras tag the test specs should pass.

njucjc commented 5 months ago

I have added two test specs, one of them ensures that tag schema is not used after tagging to an OCI v1.0 registry (which doesn't support referrers API). @njucjc With a proper fix in oras tag the test specs should pass.

@qweeah PTAL