search

openshift / oc-mirror

Lifecycle manager for internet-disconnected OpenShift environments
Apache License 2.0
82 stars 80 forks source link

OCPBUGS-34020: Implement fail safe / fail fast in workers #865

Closed sherine-k closed 3 weeks ago

sherine-k commented 1 month ago

Description

This PR contains:

Fixes OCPBUGS-34020

Type of change

Please delete options that are not relevant.

How Has This Been Tested?

With the following imagesetconfig, a mirror to mirror was performed.

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v2alpha1
mirror:
  additionalImages:
  - name: registry.redhat.io/ubi8/ubi:latest
  - name: quay.io/mantisnet/kernel-headers@sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068
  - name: registry.redhat.io/ubi9/ubi:latest

Expected Outcome

Although the second image is in error (unauthorized), the mirroring continues. ITMS is generated. No IDMS, no CatalogSource. ⚠️ Please note that exit code is 0 in the end. 

$ ./bin/oc-mirror --v2 -c configs_logs/isc_34020.yaml --workspace file:///home/skhoury/clid20/ docker://localhost:5000 --dest-tls-verify=false

2024/06/11 14:08:20  [WARN]   : ⚠️  --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready.
2024/06/11 14:08:20  [INFO]   : 👋 Hello, welcome to oc-mirror
2024/06/11 14:08:20  [INFO]   : ⚙️  setting up the environment for you...
2024/06/11 14:08:20  [INFO]   : 🔀 workflow mode: mirrorToMirror 
2024/06/11 14:08:20  [INFO]   : 🕵️  going to discover the necessary images...
2024/06/11 14:08:20  [INFO]   : 🔍 collecting release images...
2024/06/11 14:08:20  [INFO]   : 🔍 collecting operator images...
2024/06/11 14:08:20  [INFO]   : 🔍 collecting additional images...
2024/06/11 14:08:20  [INFO]   : 🚀 Start copying the images...
2024/06/11 14:08:22  [INFO]   : === Overall Progress -  image 1 / 3 ===
2024/06/11 14:08:22  [INFO]   :  additional image 1 / 3
2024/06/11 14:08:22  [INFO]   :  image: docker://registry.redhat.io/ubi8/ubi:latest
2024/06/11 14:08:22  [INFO]   : =======================================
2024/06/11 14:08:23  [INFO]   : === Overall Progress -  image 2 / 3 (1 errors)===
2024/06/11 14:08:23  [INFO]   :  additional image 2 / 3 (1 errors)
2024/06/11 14:08:23  [INFO]   :  image: docker://quay.io/mantisnet/kernel-headers@sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068
2024/06/11 14:08:23  [INFO]   : =================================================
2024/06/11 14:08:26  [INFO]   : === Overall Progress -  image 3 / 3 (1 errors)===
2024/06/11 14:08:26  [INFO]   :  additional image 3 / 3 (1 errors)
2024/06/11 14:08:26  [INFO]   :  image: docker://registry.redhat.io/ubi9/ubi:latest
2024/06/11 14:08:26  [INFO]   : =================================================
2024/06/11 14:08:26  [INFO]   : === Results ===
2024/06/11 14:08:26  [INFO]   : Images mirrored 2 / 3: Some additional images failed to mirror ❌ - please check the logs
2024/06/11 14:08:26  [ERROR]  : [Worker] error mirroring image docker://quay.io/mantisnet/kernel-headers@sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068 error: initializing source docker://quay.io/mantisnet/kernel-headers@sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068: reading manifest sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068 in quay.io/mantisnet/kernel-headers: unauthorized: access to the requested resource is not authorized
2024/06/11 14:08:26  [INFO]   : 📄 No images by digests were mirrored. Skipping IDMS generation.
2024/06/11 14:08:26  [INFO]   : 📄 Generating ITMS file...
2024/06/11 14:08:26  [INFO]   : /home/skhoury/clid20/working-dir/cluster-resources/itms-oc-mirror.yaml file created
2024/06/11 14:08:26  [INFO]   : 📄 No catalogs mirrored. Skipping CatalogSource file generation.
2024/06/11 14:08:26  [INFO]   : mirror time     : 6.531477598s
2024/06/11 14:08:26  [WARN]   : [Worker] some errors occurred during the mirroring.
         Please review /home/skhoury/clid20/working-dir/logs/mirroring_errors_20240611_140826.txt for a list of mirroring errors.
         You may consider:
         * removing images or operators that cause the error from the image set config, and retrying
         * keeping the image set config (images are mandatory for you), and retrying
         * mirroring the failing images manually, if retries also fail.
2024/06/11 14:08:26  [INFO]   : 👋 Goodbye, thank you for using oc-mirror
openshift-ci-robot commented 1 month ago

@sherine-k: This pull request references Jira Issue OCPBUGS-34020, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug * bug is open, matching expected state (open) * bug target version (4.17.0) matches configured target version for branch (4.17.0) * bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact: /cc @zhouying7780

The bug has been updated to refer to the pull request using the external bug tracker.

In response to [this](https://github.com/openshift/oc-mirror/pull/865): ># Description >This PR contains: >* A fix for the integration tests, which were failing to find test data >* Implementation of fail safe vs fail fast for batch workers: > * When calling `o.Mirror.Run`, the batch now evaluates the error type, categorizing it as failSafe or failFast > * Roughly `isFailSafe` is inspired by [`isRetryable`](https://github.com/containers/common/blob/49ad520556e7eaaeaa5263639588d82683c29b5b/pkg/retry/retry.go#L63) : all errors that are retryable are considered unsafe (fail fast). > * The error returned by the `Worker` function is typed : `UnsafeError` vs `SafeError`. The `Executor` or `Delete` will resume activities if the error is of type `SafeError` > >Fixes OCPBUGS-34020 > >## Type of change > >Please delete options that are not relevant. > >- [x] Bug fix (non-breaking change which fixes an issue) >- [ ] New feature (non-breaking change which adds functionality) >- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) >- [ ] This change requires a documentation update > ># How Has This Been Tested? > >With the following imagesetconfig, a mirror to disk was performed. >```yaml >kind: ImageSetConfiguration >apiVersion: mirror.openshift.io/v2alpha1 >mirror: > additionalImages: > - name: registry.redhat.io/ubi8/ubi:sherine > - name: registry.redhat.io/ubi8/ubi:latest >``` > >## Expected Outcome >Although the first tag doesn't exit (manifest unknown), the tar file mirror_000001.tar is generated. > ⚠️ **Please note that exit code is 1 in the end. ==> Do you think this is the expected behavior?** >``` >$ ./bin/oc-mirror --v2 -c configs_logs/isc_34020.yaml file:///home/skhoury/clid20/ > >2024/06/04 08:57:48 [WARN] : ⚠️ --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready. >2024/06/04 08:57:48 [INFO] : 👋 Hello, welcome to oc-mirror >2024/06/04 08:57:48 [INFO] : ⚙️ setting up the environment for you... >2024/06/04 08:57:48 [INFO] : 🔀 workflow mode: mirrorToDisk >2024/06/04 08:57:48 [INFO] : 🕵️ going to discover the necessary images... >2024/06/04 08:57:48 [INFO] : 🔍 collecting release images... >2024/06/04 08:57:48 [INFO] : 🔍 collecting operator images... >2024/06/04 08:57:48 [INFO] : 🔍 collecting additional images... >2024/06/04 08:57:48 [INFO] : 🚀 Start copying the images... >2024/06/04 08:57:48 [INFO] : === Overall Progress - copying image 1 / 2 === >2024/06/04 08:57:48 [INFO] : copying release image 0 / 0 >2024/06/04 08:57:48 [INFO] : copying operator image 0 / 0 >2024/06/04 08:57:48 [INFO] : copying additional image 1 / 2 >2024/06/04 08:57:48 [INFO] : ============================================== >2024/06/04 08:57:48 [INFO] : copying image: docker://registry.redhat.io/ubi8/ubi:sherine >2024/06/04 08:57:49 [INFO] : === Overall Progress - copying image 2 / 2 (1 errors)=== >2024/06/04 08:57:49 [INFO] : copying release image 0 / 0 >2024/06/04 08:57:49 [INFO] : copying operator image 0 / 0 >2024/06/04 08:57:49 [INFO] : copying additional image 2 / 2 (1 errors) >2024/06/04 08:57:49 [INFO] : ======================================================== >2024/06/04 08:57:49 [INFO] : copying image: docker://registry.redhat.io/ubi8/ubi:latest >2024/06/04 08:57:52 [INFO] : === Results === >2024/06/04 08:57:52 [INFO] : All release images mirrored successfully 0 / 0 ✅ >2024/06/04 08:57:52 [INFO] : All operator images mirrored successfully 0 / 0 ✅ >2024/06/04 08:57:52 [INFO] : Images mirrored 1 / 2: Some additional images failed to mirror ❌ - please check the logs >2024/06/04 08:57:52 [ERROR] : [Worker] error mirroring image docker://registry.redhat.io/ubi8/ubi:sherine error: initializing source docker://registry.redhat.io/ubi8/ubi:sherine: reading manifest sherine in registry.redhat.io/ubi8/ubi: manifest unknown >2024/06/04 08:57:52 [INFO] : 📦 Preparing the tarball archive... >2024/06/04 08:58:22 [INFO] : 👋 Goodbye, thank you for using oc-mirror >2024/06/04 08:58:22 [ERROR] : unable to add image blobs to the archive : unable to find blobs corresponding to docker://localhost:55000/ubi8/ubi:sherine: reading manifest sherine in localhost:55000/ubi8/ubi: manifest unknown >``` > > Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
openshift-ci[bot] commented 1 month ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sherine-k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/openshift/oc-mirror/blob/main/OWNERS)~~ [sherine-k] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
openshift-ci-robot commented 1 month ago

@sherine-k: This pull request references Jira Issue OCPBUGS-34020, which is valid.

3 validation(s) were run on this bug * bug is open, matching expected state (open) * bug target version (4.17.0) matches configured target version for branch (4.17.0) * bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact: /cc @zhouying7780

The bug has been updated to refer to the pull request using the external bug tracker.

In response to [this](https://github.com/openshift/oc-mirror/pull/865): ># Description >This PR contains: >* A fix for the integration tests, which were failing to find test data >* Implementation of fail safe vs fail fast for batch workers: > * When calling `o.Mirror.Run`, the batch now evaluates the error type, categorizing it as failSafe or failFast > * Roughly `isFailSafe` is inspired by [`isRetryable`](https://github.com/containers/common/blob/49ad520556e7eaaeaa5263639588d82683c29b5b/pkg/retry/retry.go#L63) : all errors that are retryable are considered unsafe (fail fast). > * The error returned by the `Worker` function is typed : `UnsafeError` vs `SafeError`. The `Executor` or `Delete` will resume activities if the error is of type `SafeError` > >Fixes OCPBUGS-34020 > >## Type of change > >Please delete options that are not relevant. > >- [x] Bug fix (non-breaking change which fixes an issue) >- [ ] New feature (non-breaking change which adds functionality) >- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) >- [ ] This change requires a documentation update > ># How Has This Been Tested? > >With the following imagesetconfig, a mirror to disk was performed. >```yaml >kind: ImageSetConfiguration >apiVersion: mirror.openshift.io/v2alpha1 >mirror: > additionalImages: > - name: registry.redhat.io/ubi8/ubi:sherine > - name: registry.redhat.io/ubi8/ubi:latest >``` > >## Expected Outcome >Although the first tag doesn't exit (manifest unknown), the tar file mirror_000001.tar is generated. > ⚠️ **Please note that exit code is 0 in the end. ** >``` >$ ./bin/oc-mirror --v2 -c configs_logs/isc_34020.yaml file:///home/skhoury/clid20/ > >2024/06/04 08:57:48 [WARN] : ⚠️ --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready. >2024/06/04 08:57:48 [INFO] : 👋 Hello, welcome to oc-mirror >2024/06/04 08:57:48 [INFO] : ⚙️ setting up the environment for you... >2024/06/04 08:57:48 [INFO] : 🔀 workflow mode: mirrorToDisk >2024/06/04 08:57:48 [INFO] : 🕵️ going to discover the necessary images... >2024/06/04 08:57:48 [INFO] : 🔍 collecting release images... >2024/06/04 08:57:48 [INFO] : 🔍 collecting operator images... >2024/06/04 08:57:48 [INFO] : 🔍 collecting additional images... >2024/06/04 08:57:48 [INFO] : 🚀 Start copying the images... >2024/06/04 08:57:48 [INFO] : === Overall Progress - copying image 1 / 2 === >2024/06/04 08:57:48 [INFO] : copying release image 0 / 0 >2024/06/04 08:57:48 [INFO] : copying operator image 0 / 0 >2024/06/04 08:57:48 [INFO] : copying additional image 1 / 2 >2024/06/04 08:57:48 [INFO] : ============================================== >2024/06/04 08:57:48 [INFO] : copying image: docker://registry.redhat.io/ubi8/ubi:sherine >2024/06/04 08:57:49 [INFO] : === Overall Progress - copying image 2 / 2 (1 errors)=== >2024/06/04 08:57:49 [INFO] : copying release image 0 / 0 >2024/06/04 08:57:49 [INFO] : copying operator image 0 / 0 >2024/06/04 08:57:49 [INFO] : copying additional image 2 / 2 (1 errors) >2024/06/04 08:57:49 [INFO] : ======================================================== >2024/06/04 08:57:49 [INFO] : copying image: docker://registry.redhat.io/ubi8/ubi:latest >2024/06/04 08:57:52 [INFO] : === Results === >2024/06/04 08:57:52 [INFO] : All release images mirrored successfully 0 / 0 ✅ >2024/06/04 08:57:52 [INFO] : All operator images mirrored successfully 0 / 0 ✅ >2024/06/04 08:57:52 [INFO] : Images mirrored 1 / 2: Some additional images failed to mirror ❌ - please check the logs >2024/06/04 08:57:52 [ERROR] : [Worker] error mirroring image docker://registry.redhat.io/ubi8/ubi:sherine error: initializing source docker://registry.redhat.io/ubi8/ubi:sherine: reading manifest sherine in registry.redhat.io/ubi8/ubi: manifest unknown >2024/06/04 08:57:52 [INFO] : 📦 Preparing the tarball archive... >2024/06/04 08:58:22 [INFO] : 👋 Goodbye, thank you for using oc-mirror >``` > > Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
sherine-k commented 1 month ago

/retest

sherine-k commented 1 month ago

/cherrypick release-4.16

openshift-cherrypick-robot commented 1 month ago

@sherine-k: once the present PR merges, I will cherry-pick it on top of release-4.16 in a new PR and assign it to you.

In response to [this](https://github.com/openshift/oc-mirror/pull/865#issuecomment-2150081360): >/cherrypick release-4.16 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
sherine-k commented 4 weeks ago

/retest-required

sherine-k commented 3 weeks ago

/retest

kasturinarra commented 3 weeks ago

/lgtm

kasturinarra commented 3 weeks ago

/label qe-approved

openshift-ci-robot commented 3 weeks ago

@sherine-k: Jira Issue OCPBUGS-34020: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-34020 has been moved to the MODIFIED state.

In response to [this](https://github.com/openshift/oc-mirror/pull/865): ># Description >This PR contains: >* A fix for the integration tests, which were failing to find test data >* Rebased after #864 >* Modification of the way we log progress in batch mirroring: > * image reference is shown, > * progress on release/operator/additional is only shown if there are images of that type in the imageset > * code: handling the counters for progress and logging it is deported to separate function >* Implementation of fail safe vs fail fast for batch workers: > * When calling `o.Mirror.Run`, the batch now evaluates the image type: > * For error on release images, the batch will immediately fail and return to the caller: no IDMS or ITMS or catalogSource are to be generated > * For error on any other type of image, the batch continues, reports a `SafeError` to caller. The executor will carry on generating cluster resources. > * The error returned by the `Worker` function is typed : `UnsafeError` vs `SafeError`. The `Executor` or `Delete` will resume activities if the error is of type `SafeError` > * `isFailSafe` is commented: this function was originally used to determine if errors were `UnsafeError` or `SafeError`. Its use is deferred to later (see OCPBUGS-34020 comments for decision details). > * It is inspired by [`isRetryable`](https://github.com/containers/common/blob/49ad520556e7eaaeaa5263639588d82683c29b5b/pkg/retry/retry.go#L63) : all errors that are retryable are considered unsafe (fail fast). > >Fixes OCPBUGS-34020 > >## Type of change > >Please delete options that are not relevant. > >- [x] Bug fix (non-breaking change which fixes an issue) >- [ ] New feature (non-breaking change which adds functionality) >- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) >- [ ] This change requires a documentation update > ># How Has This Been Tested? > >With the following imagesetconfig, a mirror to mirror was performed. >```yaml >kind: ImageSetConfiguration >apiVersion: mirror.openshift.io/v2alpha1 >mirror: > additionalImages: > - name: registry.redhat.io/ubi8/ubi:latest > - name: quay.io/mantisnet/kernel-headers@sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068 > - name: registry.redhat.io/ubi9/ubi:latest >``` > >## Expected Outcome >Although the second image is in error (unauthorized), the mirroring continues. ITMS is generated. No IDMS, no CatalogSource. > ⚠️ **Please note that exit code is 0 in the end. ** >``` >$ ./bin/oc-mirror --v2 -c configs_logs/isc_34020.yaml --workspace file:///home/skhoury/clid20/ docker://localhost:5000 --dest-tls-verify=false > >2024/06/11 14:08:20 [WARN] : ⚠️ --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready. >2024/06/11 14:08:20 [INFO] : 👋 Hello, welcome to oc-mirror >2024/06/11 14:08:20 [INFO] : ⚙️ setting up the environment for you... >2024/06/11 14:08:20 [INFO] : 🔀 workflow mode: mirrorToMirror >2024/06/11 14:08:20 [INFO] : 🕵️ going to discover the necessary images... >2024/06/11 14:08:20 [INFO] : 🔍 collecting release images... >2024/06/11 14:08:20 [INFO] : 🔍 collecting operator images... >2024/06/11 14:08:20 [INFO] : 🔍 collecting additional images... >2024/06/11 14:08:20 [INFO] : 🚀 Start copying the images... >2024/06/11 14:08:22 [INFO] : === Overall Progress - image 1 / 3 === >2024/06/11 14:08:22 [INFO] : additional image 1 / 3 >2024/06/11 14:08:22 [INFO] : image: docker://registry.redhat.io/ubi8/ubi:latest >2024/06/11 14:08:22 [INFO] : ======================================= >2024/06/11 14:08:23 [INFO] : === Overall Progress - image 2 / 3 (1 errors)=== >2024/06/11 14:08:23 [INFO] : additional image 2 / 3 (1 errors) >2024/06/11 14:08:23 [INFO] : image: docker://quay.io/mantisnet/kernel-headers@sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068 >2024/06/11 14:08:23 [INFO] : ================================================= >2024/06/11 14:08:26 [INFO] : === Overall Progress - image 3 / 3 (1 errors)=== >2024/06/11 14:08:26 [INFO] : additional image 3 / 3 (1 errors) >2024/06/11 14:08:26 [INFO] : image: docker://registry.redhat.io/ubi9/ubi:latest >2024/06/11 14:08:26 [INFO] : ================================================= >2024/06/11 14:08:26 [INFO] : === Results === >2024/06/11 14:08:26 [INFO] : Images mirrored 2 / 3: Some additional images failed to mirror ❌ - please check the logs >2024/06/11 14:08:26 [ERROR] : [Worker] error mirroring image docker://quay.io/mantisnet/kernel-headers@sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068 error: initializing source docker://quay.io/mantisnet/kernel-headers@sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068: reading manifest sha256:242e837af831b6eed98d522b6bae7416928b9fbe451fb6e02ffa678686771068 in quay.io/mantisnet/kernel-headers: unauthorized: access to the requested resource is not authorized >2024/06/11 14:08:26 [INFO] : 📄 No images by digests were mirrored. Skipping IDMS generation. >2024/06/11 14:08:26 [INFO] : 📄 Generating ITMS file... >2024/06/11 14:08:26 [INFO] : /home/skhoury/clid20/working-dir/cluster-resources/itms-oc-mirror.yaml file created >2024/06/11 14:08:26 [INFO] : 📄 No catalogs mirrored. Skipping CatalogSource file generation. >2024/06/11 14:08:26 [INFO] : mirror time : 6.531477598s >2024/06/11 14:08:26 [WARN] : [Worker] some errors occurred during the mirroring. > Please review /home/skhoury/clid20/working-dir/logs/mirroring_errors_20240611_140826.txt for a list of mirroring errors. > You may consider: > * removing images or operators that cause the error from the image set config, and retrying > * keeping the image set config (images are mandatory for you), and retrying > * mirroring the failing images manually, if retries also fail. >2024/06/11 14:08:26 [INFO] : 👋 Goodbye, thank you for using oc-mirror >``` > > Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
openshift-cherrypick-robot commented 3 weeks ago

@sherine-k: new pull request created: #879

In response to [this](https://github.com/openshift/oc-mirror/pull/865#issuecomment-2150081360): >/cherrypick release-4.16 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
openshift-bot commented 3 weeks ago

[ART PR BUILD NOTIFIER]

This PR has been included in build oc-mirror-plugin-container-v4.17.0-202406131343.p0.g157eb08.assembly.stream.el9 for distgit oc-mirror-plugin. All builds following this will include this PR.