search

openshift / oc-mirror

Lifecycle manager for internet-disconnected OpenShift environments
Apache License 2.0
82 stars 80 forks source link

OCPBUGS-36214: Fix V2 DiskToMirror should not require internet access #882

Closed sherine-k closed 4 days ago

sherine-k commented 1 week ago

Description

This PR fixes the errors encountered while DiskToMirror in a disconnected environment. The issue arises because

When diskToMirror is happening on a fully disconnected environment, there is no access to the source registry. This is why the error was happening.

With this fix, oc-mirror interrogates the local cache in order to determine this digest.

Fixes # OCPBUGS-36214

Type of change

Please delete options that are not relevant.

How Has This Been Tested?

Use cases:

For each of the above use cases the 3 workflows were performed:

Expected Outcome

All workflows should pass.

openshift-ci-robot commented 1 week ago

@sherine-k: No Jira issue with key OCPBUGS-362214 exists in the tracker at https://issues.redhat.com/. Once a valid jira issue is referenced in the title of this pull request, request a refresh with /jira refresh.

In response to [this](https://github.com/openshift/oc-mirror/pull/882): ># Description > >Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. > >Fixes # (issue) > >## Type of change > >Please delete options that are not relevant. > >- [ ] Bug fix (non-breaking change which fixes an issue) >- [ ] New feature (non-breaking change which adds functionality) >- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) >- [ ] This change requires a documentation update > ># How Has This Been Tested? > >Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration > >## Expected Outcome >Please describe the outcome expected from the tests Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
openshift-ci[bot] commented 1 week ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sherine-k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/openshift/oc-mirror/blob/main/OWNERS)~~ [sherine-k] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
openshift-ci-robot commented 1 week ago

@sherine-k: No Jira issue with key OCPBUGS-362214 exists in the tracker at https://issues.redhat.com/. Once a valid jira issue is referenced in the title of this pull request, request a refresh with /jira refresh.

In response to [this](https://github.com/openshift/oc-mirror/pull/882): ># Description >This PR fixes the errors encountered while DiskToMirror in a disconnected environment. >The issue arises because >* when in MirrorToDisk, the catalog image(working-dir/operator-images) and contents (working-dir/hold-images) are stored under a subfolder corresponding to the digest of the image. >* Then, while doing diskToMirror, oc-mirror attempts to call the source registry to resolve the image tag to a digest in order to find the corresponding subfolder on disk. > >When diskToMirror is happening on a fully disconnected environment, there is no access to the source registry. This is why the error was happening. > >With this fix, oc-mirror interrogates the local cache in order to determine this digest. > >Fixes # [OCPBUGS-36214](https://issues.redhat.com/browse/OCPBUGS-36214) > >## Type of change > >Please delete options that are not relevant. > >- [x] Bug fix (non-breaking change which fixes an issue) >- [ ] New feature (non-breaking change which adds functionality) >- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) >- [ ] This change requires a documentation update > ># How Has This Been Tested? >**Use cases**: >* catalog is a remote reference > * without targetCatalog nor targetTag > * with TargetCatalog > * with TargetTag > * with both TargetCatalog and TargetTag >* catalog is a OCI local reference > * without targetCatalog nor targetTag > * with TargetCatalog > * with TargetTag > * with both TargetCatalog and TargetTag > >For each of the above use cases the 3 workflows were performed: >- mirror to disk twice: > - first time starting by `rm -fr $WORKING_DIR/hold-operator/* $WORKING_DIR/operator-images/*` (forcing redownload) > - and second time without (using cached image) >- disk to mirror without internet >- mirror to mirror > >## Expected Outcome >All workflows should pass. Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
openshift-ci-robot commented 1 week ago

@sherine-k: This pull request references Jira Issue OCPBUGS-36214, which is invalid:

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to [this](https://github.com/openshift/oc-mirror/pull/882): ># Description >This PR fixes the errors encountered while DiskToMirror in a disconnected environment. >The issue arises because >* when in MirrorToDisk, the catalog image(working-dir/operator-images) and contents (working-dir/hold-images) are stored under a subfolder corresponding to the digest of the image. >* Then, while doing diskToMirror, oc-mirror attempts to call the source registry to resolve the image tag to a digest in order to find the corresponding subfolder on disk. > >When diskToMirror is happening on a fully disconnected environment, there is no access to the source registry. This is why the error was happening. > >With this fix, oc-mirror interrogates the local cache in order to determine this digest. > >Fixes # [OCPBUGS-36214](https://issues.redhat.com/browse/OCPBUGS-36214) > >## Type of change > >Please delete options that are not relevant. > >- [x] Bug fix (non-breaking change which fixes an issue) >- [ ] New feature (non-breaking change which adds functionality) >- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) >- [ ] This change requires a documentation update > ># How Has This Been Tested? >**Use cases**: >* catalog is a remote reference > * without targetCatalog nor targetTag > * with TargetCatalog > * with TargetTag > * with both TargetCatalog and TargetTag >* catalog is a OCI local reference > * without targetCatalog nor targetTag > * with TargetCatalog > * with TargetTag > * with both TargetCatalog and TargetTag > >For each of the above use cases the 3 workflows were performed: >- mirror to disk twice: > - first time starting by `rm -fr $WORKING_DIR/hold-operator/* $WORKING_DIR/operator-images/*` (forcing redownload) > - and second time without (using cached image) >- disk to mirror without internet >- mirror to mirror > >## Expected Outcome >All workflows should pass. Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
openshift-ci[bot] commented 1 week ago

@sherine-k: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
sherine-k commented 1 week ago

/jira refresh

openshift-ci-robot commented 1 week ago

@sherine-k: This pull request references Jira Issue OCPBUGS-36214, which is invalid:

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to [this](https://github.com/openshift/oc-mirror/pull/882#issuecomment-2196862632): >/jira refresh Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
sherine-k commented 1 week ago

/cherry-pick release-4.16

openshift-cherrypick-robot commented 1 week ago

@sherine-k: once the present PR merges, I will cherry-pick it on top of release-4.16 in a new PR and assign it to you.

In response to [this](https://github.com/openshift/oc-mirror/pull/882#issuecomment-2196863063): >/cherry-pick release-4.16 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
zhouying7780 commented 5 days ago

/label qe-approved

sherine-k commented 4 days ago

/jira refresh

openshift-ci-robot commented 4 days ago

@sherine-k: This pull request references Jira Issue OCPBUGS-36214, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug * bug is open, matching expected state (open) * bug target version (4.17.0) matches configured target version for branch (4.17.0) * bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact: /cc @zhouying7780

In response to [this](https://github.com/openshift/oc-mirror/pull/882#issuecomment-2200074151): >/jira refresh Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
lmzuccarelli commented 4 days ago

/lgtm

openshift-ci-robot commented 4 days ago

@sherine-k: Jira Issue OCPBUGS-36214: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-36214 has been moved to the MODIFIED state.

In response to [this](https://github.com/openshift/oc-mirror/pull/882): ># Description >This PR fixes the errors encountered while DiskToMirror in a disconnected environment. >The issue arises because >* when in MirrorToDisk, the catalog image(working-dir/operator-images) and contents (working-dir/hold-images) are stored under a subfolder corresponding to the digest of the image. >* Then, while doing diskToMirror, oc-mirror attempts to call the source registry to resolve the image tag to a digest in order to find the corresponding subfolder on disk. > >When diskToMirror is happening on a fully disconnected environment, there is no access to the source registry. This is why the error was happening. > >With this fix, oc-mirror interrogates the local cache in order to determine this digest. > >Fixes # [OCPBUGS-36214](https://issues.redhat.com/browse/OCPBUGS-36214) > >## Type of change > >Please delete options that are not relevant. > >- [x] Bug fix (non-breaking change which fixes an issue) >- [ ] New feature (non-breaking change which adds functionality) >- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) >- [ ] This change requires a documentation update > ># How Has This Been Tested? >**Use cases**: >* catalog is a remote reference > * without targetCatalog nor targetTag > * with TargetCatalog > * with TargetTag > * with both TargetCatalog and TargetTag >* catalog is a OCI local reference > * without targetCatalog nor targetTag > * with TargetCatalog > * with TargetTag > * with both TargetCatalog and TargetTag > >For each of the above use cases the 3 workflows were performed: >- mirror to disk twice: > - first time starting by `rm -fr $WORKING_DIR/hold-operator/* $WORKING_DIR/operator-images/*` (forcing redownload) > - and second time without (using cached image) >- disk to mirror without internet >- mirror to mirror > >## Expected Outcome >All workflows should pass. Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
openshift-cherrypick-robot commented 4 days ago

@sherine-k: new pull request created: #884

In response to [this](https://github.com/openshift/oc-mirror/pull/882#issuecomment-2196863063): >/cherry-pick release-4.16 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
openshift-bot commented 3 days ago

[ART PR BUILD NOTIFIER]

This PR has been included in build oc-mirror-plugin-container-v4.17.0-202407020721.p0.g357f9ff.assembly.stream.el9 for distgit oc-mirror-plugin. All builds following this will include this PR.