search

openshift / machine-config-operator

Apache License 2.0
244 stars 396 forks source link

OCPBUGS-36344: Add CIP relevant mirrors to sigstore attachement cfg #4449

Closed QiWang19 closed 1 week ago

QiWang19 commented 2 weeks ago

Close: #4446 - What I did

- How to verify it

Cluster 4.17.0-0.ci.test-2024-07-08-173847 has default ICSP:

$ oc describe imagecontentsourcepolicy/image-policy
Name:         image-policy
Namespace:    
Labels:       <none>
Annotations:  <none>
API Version:  operator.openshift.io/v1alpha1
Kind:         ImageContentSourcePolicy
Metadata:
  Creation Timestamp:  2024-07-08T17:51:37Z
  Generation:          1
  Resource Version:    706
  UID:                 036f79f1-826c-459c-8adc-8b3cc0499801
Spec:
  Repository Digest Mirrors:
    Mirrors:
      quayio-pull-through-cache-us-east-2-ci.apps.ci.l2s4.p1.openshiftapps.com
    Source:  quay.io
Events:      <none>

Apply CIP:

apiVersion: config.openshift.io/v1alpha1
kind: ClusterImagePolicy
metadata:
  name: openshift
  annotations:
    kubernetes.io/description: Require Red Hat signatures for quay.io/openshift-release-dev/ocp-release container images.
    exclude.release.openshift.io/internal-openshift-hosted: "true"
    include.release.openshift.io/self-managed-high-availability: "true"
    release.openshift.io/feature-set: TechPreviewNoUpgrade
spec:
  scopes:
  - quay.io/openshift-release-dev/ocp-release
  policy:
    rootOfTrust:
      policyType: PublicKey
      publicKey:
        keyData: LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUEzQzJlVGdJQUo3aGxveDdDSCtIcE1qdDEvbW5lYXcyejlHdE9NUmlSaEgya09ZalRadGVLSEtnWUJHcGViajRBcUpWYnVRaWJYZTZKYVFHQUFER0VOZXozTldsVXpCby9FUUEwaXJDRnN6dlhVbTE2cWFZMG8zOUZpbWpsVVovaG1VNVljSHhxMzR2OTh4bGtRbUVxekowR0VJMzNtWTFMbWFEM3ZhYmd3WWcwb3lzSTk1Z1V1Tk81TmdZUHA4WDREaFNoSmtyVEl5dDJLTEhYWW5BMExzOEJlbG9PWVJlTnJhZmxKRHNzaE5VRFh4MDJhQVZSd2RjMXhJUDArRTlZaTY1ZE4zKzlReVhEOUZ6K3MrTDNjZzh3bDdZd3ZZb1Z2NDhndklmTHlJbjJUaHY2Uzk2R0V6bXBoazRjWDBIeitnUkdocWpyajU4U2hSZzlteitrcnVhR0VuVGcyS3BWR0gzd3I4Z09UdUFZMmtqMnY1YWhnZWt4V1pFN05vazNiNTBKNEpnYXlpSnVSL2R0cmFQMWVMMjlFMG52akdsMXptUXlGNlZnNGdIVXYwaktrcnJ2QUQ4c1dNY2NBS00zbXNXU01uRVpOTnljTTRITlNobGNReG5xU1lFSXR6MGZjajdYamtKbnAxME51Z2lVWlNLeVNXOHc0R3hTaFNraGRGbzByRDlkVElRZkJoeS91ZHRQWUkrK2VoK243QTV2UVV4Wk5BTmZqOUhRbC81Z3lFbFV6TTJOekJ2RHpHellSNVdVZEVEaDlJQ1I4ZlFpMVIxNUtZU0h2Tlc3RW5ucDdZT2d5dmtoSkdwRU5PQkF3c1pLMUhhMkJZYXZMMk05NDJzSkhxOUQ1eEsrZyszQU81eXp6V2NqaUFDMWU4RURPcUVpY01Ud05LOENBd0VBQVE9PQotLS0tLUVORCBQVUJMSUMgS0VZLS0tLS0K

Pull from mirror, check the log: Looking for sigstore attachments

sh-5.1# crictl pull quay.io/openshift-release-dev/ocp-release@sha256:c17d4489c1b283ee71c76dda559e66a546e16b208a57eb156ef38fb30098903a
E0708 19:53:07.692735    7828 remote_image.go:180] "PullImage from image service failed" err="rpc error: code = Unknown desc = SignatureValidationFailed: Source image rejected: Signature for identity quay.io/openshift-release-dev/ocp-release is not accepted" image="quay.io/openshift-release-dev/ocp-release@sha256:c17d4489c1b283ee71c76dda559e66a546e16b208a57eb156ef38fb30098903a"
FATA[0000] pulling image: SignatureValidationFailed: Source image rejected: Signature for identity quay.io/openshift-release-dev/ocp-release is not accepted 
sh-5.1# journalctl -u crio --since "1 minute ago"
Jul 08 19:53:07 ip-10-0-90-119 crio[2147]: time="2024-07-08 19:53:07.124789735Z" level=debug msg="Looking for sigstore attachments in quayio-pull-through-cache-us-east-2-ci.apps.ci.l2s4.p1.openshiftapps.com/openshift-release-dev/ocp-release:sha256-c17d...

- Description for the changelog

Add icsp/idms/itms mirrors of CIP scope to /etc/containers/registries.d, so sigstore attachment will be used during the image pull and verification.

openshift-ci[bot] commented 2 weeks ago

Skipping CI for Draft Pull Request. If you want CI signal for your change, please convert it to an actual PR. You can still manually trigger a test run with /test all

openshift-ci-robot commented 2 weeks ago

@QiWang19: This pull request references Jira Issue OCPBUGS-36344, which is invalid:

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to [this](https://github.com/openshift/machine-config-operator/pull/4449): > > >**- What I did** > >**- How to verify it** > >**- Description for the changelog** > > Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Fmachine-config-operator). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
QiWang19 commented 2 weeks ago

/jira refresh

openshift-ci-robot commented 2 weeks ago

@QiWang19: This pull request references Jira Issue OCPBUGS-36344, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug * bug is open, matching expected state (open) * bug target version (4.17.0) matches configured target version for branch (4.17.0) * bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (schoudha@redhat.com), skipping review request.

In response to [this](https://github.com/openshift/machine-config-operator/pull/4449#issuecomment-2206674690): >/jira refresh Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Fmachine-config-operator). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
QiWang19 commented 2 weeks ago

/test all

openshift-ci-robot commented 2 weeks ago

@QiWang19: This pull request references Jira Issue OCPBUGS-36344, which is valid.

3 validation(s) were run on this bug * bug is open, matching expected state (open) * bug target version (4.17.0) matches configured target version for branch (4.17.0) * bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (schoudha@redhat.com), skipping review request.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to [this](https://github.com/openshift/machine-config-operator/pull/4449): > >Close: #4446 >**- What I did** > >**- How to verify it** > > > >**- Description for the changelog** > >Add icsp/idms/itms mirrors of CIP scope to /etc/containers/registries.d, so sigstore attachment will be used during the image pull and verification. Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Fmachine-config-operator). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
QiWang19 commented 2 weeks ago

@mtrmac could you review?

mtrmac commented 2 weeks ago

Arguably, it might not be strictly necessary to be precise in the use-sigstore-attachments configuration, the performance impact of enabling that unnecessarily is a few HTTP round-trips per image. OTOH it does add some interoperability risk — if we don’t correctly recognize the registry’s response as “sigstore image not found”, that can cause the whole pull to fail.

So, I think, at the very least, it should not be hard-coded enabled via default-docker, at least for now; using somewhat wider scopes within a registry which is used to fetch sigstore-signed images is not ideal but might be acceptable if we were under time pressure.

QiWang19 commented 1 week ago

@mtrmac Could you review?
I think we can move the code to runtime-utils/pkg/registries in the follow-up PRs since we are under time pressure. What do you think?

mtrmac commented 1 week ago

/lgtm

mtrmac commented 1 week ago

The unit test failure seems not to be obviously related, but I didn’t investigate beyond reading the backtrace.

QiWang19 commented 1 week ago

/test unit

QiWang19 commented 1 week ago

/assign @saschagrunert Could you approve?

openshift-ci[bot] commented 1 week ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mtrmac, QiWang19, saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[pkg/controller/container-runtime-config/OWNERS](https://github.com/openshift/machine-config-operator/blob/master/pkg/controller/container-runtime-config/OWNERS)~~ [saschagrunert] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
openshift-ci[bot] commented 1 week ago

@QiWang19: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-gcp-op-techpreview 861d9aff147e97f0b20fc5eb532ae6f7589a6b04 link false /test e2e-gcp-op-techpreview

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
openshift-ci-robot commented 1 week ago

@QiWang19: Jira Issue OCPBUGS-36344: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-36344 has been moved to the MODIFIED state.

In response to [this](https://github.com/openshift/machine-config-operator/pull/4449): > >Close: #4446 >**- What I did** > >**- How to verify it** > >Cluster 4.17.0-0.ci.test-2024-07-08-173847 has default ICSP: >``` >$ oc describe imagecontentsourcepolicy/image-policy >Name: image-policy >Namespace: >Labels: >Annotations: >API Version: operator.openshift.io/v1alpha1 >Kind: ImageContentSourcePolicy >Metadata: > Creation Timestamp: 2024-07-08T17:51:37Z > Generation: 1 > Resource Version: 706 > UID: 036f79f1-826c-459c-8adc-8b3cc0499801 >Spec: > Repository Digest Mirrors: > Mirrors: > quayio-pull-through-cache-us-east-2-ci.apps.ci.l2s4.p1.openshiftapps.com > Source: quay.io >Events: >``` >Apply CIP: >``` >apiVersion: config.openshift.io/v1alpha1 >kind: ClusterImagePolicy >metadata: > name: openshift > annotations: > kubernetes.io/description: Require Red Hat signatures for quay.io/openshift-release-dev/ocp-release container images. > exclude.release.openshift.io/internal-openshift-hosted: "true" > include.release.openshift.io/self-managed-high-availability: "true" > release.openshift.io/feature-set: TechPreviewNoUpgrade >spec: > scopes: > - quay.io/openshift-release-dev/ocp-release > policy: > rootOfTrust: > policyType: PublicKey > publicKey: > keyData: LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUEzQzJlVGdJQUo3aGxveDdDSCtIcE1qdDEvbW5lYXcyejlHdE9NUmlSaEgya09ZalRadGVLSEtnWUJHcGViajRBcUpWYnVRaWJYZTZKYVFHQUFER0VOZXozTldsVXpCby9FUUEwaXJDRnN6dlhVbTE2cWFZMG8zOUZpbWpsVVovaG1VNVljSHhxMzR2OTh4bGtRbUVxekowR0VJMzNtWTFMbWFEM3ZhYmd3WWcwb3lzSTk1Z1V1Tk81TmdZUHA4WDREaFNoSmtyVEl5dDJLTEhYWW5BMExzOEJlbG9PWVJlTnJhZmxKRHNzaE5VRFh4MDJhQVZSd2RjMXhJUDArRTlZaTY1ZE4zKzlReVhEOUZ6K3MrTDNjZzh3bDdZd3ZZb1Z2NDhndklmTHlJbjJUaHY2Uzk2R0V6bXBoazRjWDBIeitnUkdocWpyajU4U2hSZzlteitrcnVhR0VuVGcyS3BWR0gzd3I4Z09UdUFZMmtqMnY1YWhnZWt4V1pFN05vazNiNTBKNEpnYXlpSnVSL2R0cmFQMWVMMjlFMG52akdsMXptUXlGNlZnNGdIVXYwaktrcnJ2QUQ4c1dNY2NBS00zbXNXU01uRVpOTnljTTRITlNobGNReG5xU1lFSXR6MGZjajdYamtKbnAxME51Z2lVWlNLeVNXOHc0R3hTaFNraGRGbzByRDlkVElRZkJoeS91ZHRQWUkrK2VoK243QTV2UVV4Wk5BTmZqOUhRbC81Z3lFbFV6TTJOekJ2RHpHellSNVdVZEVEaDlJQ1I4ZlFpMVIxNUtZU0h2Tlc3RW5ucDdZT2d5dmtoSkdwRU5PQkF3c1pLMUhhMkJZYXZMMk05NDJzSkhxOUQ1eEsrZyszQU81eXp6V2NqaUFDMWU4RURPcUVpY01Ud05LOENBd0VBQVE9PQotLS0tLUVORCBQVUJMSUMgS0VZLS0tLS0K >``` >Pull from mirror, check the log: `Looking for sigstore attachments` >``` >sh-5.1# crictl pull quay.io/openshift-release-dev/ocp-release@sha256:c17d4489c1b283ee71c76dda559e66a546e16b208a57eb156ef38fb30098903a >E0708 19:53:07.692735 7828 remote_image.go:180] "PullImage from image service failed" err="rpc error: code = Unknown desc = SignatureValidationFailed: Source image rejected: Signature for identity quay.io/openshift-release-dev/ocp-release is not accepted" image="quay.io/openshift-release-dev/ocp-release@sha256:c17d4489c1b283ee71c76dda559e66a546e16b208a57eb156ef38fb30098903a" >FATA[0000] pulling image: SignatureValidationFailed: Source image rejected: Signature for identity quay.io/openshift-release-dev/ocp-release is not accepted >sh-5.1# journalctl -u crio --since "1 minute ago" >Jul 08 19:53:07 ip-10-0-90-119 crio[2147]: time="2024-07-08 19:53:07.124789735Z" level=debug msg="Looking for sigstore attachments in quayio-pull-through-cache-us-east-2-ci.apps.ci.l2s4.p1.openshiftapps.com/openshift-release-dev/ocp-release:sha256-c17d... >``` >**- Description for the changelog** > >Add icsp/idms/itms mirrors of CIP scope to /etc/containers/registries.d, so sigstore attachment will be used during the image pull and verification. Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Fmachine-config-operator). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
openshift-bot commented 1 week ago

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-machine-config-operator-container-v4.17.0-202407111341.p0.g35ce1c1.assembly.stream.el9 for distgit ose-machine-config-operator. All builds following this will include this PR.