openshift / oc-mirror

Lifecycle manager for internet-disconnected OpenShift environments
Apache License 2.0
91 stars 82 forks source link

OCPSTRAT-1439,CLID-28,CLID-170: adds feature to mirror helm charts in oc-mirror v2 #935

Closed aguidirh closed 1 month ago

aguidirh commented 1 month ago

Description

This PR adds the feature to mirror helm charts in oc-mirror v2.

Jira issues: OCPSTRAT-1439 CLID-28 CLID-170

Type of change

Please delete options that are not relevant.

How Has This Been Tested?

With the following ImageSetConfiguration

apiVersion: mirror.openshift.io/v2alpha1
kind: ImageSetConfiguration
mirror:
  helm:
    repositories:
      - name: podinfo
        url: https://stefanprodan.github.io/podinfo
        charts:
          - name: podinfo
            version: 5.0.0
      - name: sbo
        url: https://redhat-developer.github.io/service-binding-operator-helm-chart/
    # local:
    #  - name: podinfo-local
    #    path: /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/local-test-clid-28/podinfo-5.0.0.tgz

Run all flows (mirrorToDisk, diskToMirror and mirrorToMirror)

mirrorToDisk

./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/clid-28-v2.yaml file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28 --v2

diskToMirror

./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/clid-28-v2.yaml --from file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28 docker://localhost:6000 --v2

mirrorToMirror

./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/clid-28-v2.yaml --workspace file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28 docker://localhost:6000 --v2

Expected Outcome

All flows should finish successfully and the images should be in the target registry as showed below:

curl http://localhost:6000/v2/_catalog | jq
{
  "repositories": [
    "redhat-developer/servicebinding-operator",
    "stefanprodan/podinfo"
  ]
}
curl http://localhost:6000/v2/stefanprodan/podinfo/tags/list | jq
{
  "name": "stefanprodan/podinfo",
  "tags": [
    "5.0.0"
  ]
}
curl http://localhost:6000/v2/redhat-developer/servicebinding-operator/tags/list | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   801  100   801    0     0   316k      0 --:--:-- --:--:-- --:--:--  391k
{
  "name": "redhat-developer/servicebinding-operator",
  "tags": [
    "sha256-16286ac84ddd521897d92472dae857a4c18479f255b725dfb683bc72df6e0865",
    "sha256-30bf7f0f21024bb2e1e4db901b1f5e89ab56e0f3197a919d2bbb670f3fe5223a",
    "sha256-67c2a2502f59fac1e7ded9ed19b59bbd4e50f5559a13978a87ecd2283b81e067",
    "sha256-ac47f496fb7ecdcbc371f8c809fad2687ec0c35bbc8c522a7ab63b3e5ffd90ea",
    "sha256-f79f6999a15534dbe56e658caf94fc4b7afb5ceeb7b49f32a60ead06fbd7c3fc",
    "sha256-e4259939a496f292a31b5e57760196d63a8182b999164d93a446da48c4ea24eb",
    "sha256-e01016cacae84dfb6eaf7a1022130e7d95e2a8489c38d4d46e4f734848e93849",
    "sha256-de1881753e82c51b31e958fcf383cb35b0f70f6ec99d402d42243e595d00c6dd",
    "sha256-69a95c6216ead931e01e4144ae8f4fb7ab35d1f68a14c18f6860a085ccb950f5",
    "sha256-cc5aab01ddd3744510c480eb4f58b834936a833d36bec5c9c13fb40bbb06c663"
  ]
}

IDMS and ITMS should be generated as showed below:

📄 Generating IDMS file...
2024/10/02 16:10:25  [INFO]   : /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28/working-dir/cluster-resources/idms-oc-mirror.yaml file created
2024/10/02 16:10:25  [INFO]   : 📄 Generating ITMS file...
2024/10/02 16:10:25  [INFO]   : /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28/working-dir/cluster-resources/itms-oc-mirror.yaml file created

IDMS

---
apiVersion: config.openshift.io/v1
kind: ImageDigestMirrorSet
metadata:
  name: idms-generic-0
spec:
  imageDigestMirrors:
  - mirrors:
    - localhost:6000/redhat-developer
    source: quay.io/redhat-developer
status: {}

ITMS

---
apiVersion: config.openshift.io/v1
kind: ImageTagMirrorSet
metadata:
  name: itms-generic-0
spec:
  imageTagMirrors:
  - mirrors:
    - localhost:6000/stefanprodan
    source: ghcr.io/stefanprodan
status: {}
openshift-ci-robot commented 1 month ago

@aguidirh: This pull request references OCPSTRAT-1439 which is a valid jira issue.

This pull request references CLID-28 which is a valid jira issue.

This pull request references CLID-170 which is a valid jira issue.

In response to [this](https://github.com/openshift/oc-mirror/pull/935): ># Description > >This PR adds the feature to mirror helm charts in oc-mirror v2. > >Jira issues: >[OCPSTRAT-1439](https://issues.redhat.com/browse/OCPSTRAT-1439) >[CLID-28](https://issues.redhat.com/browse/CLID-28) >[CLID-170](https://issues.redhat.com/browse/CLID-170) > >## Type of change > >Please delete options that are not relevant. >- [x] New feature (non-breaking change which adds functionality) >- [x] This change requires a documentation update > ># How Has This Been Tested? > >With the following ImageSetConfiguration >``` >apiVersion: mirror.openshift.io/v2alpha1 >kind: ImageSetConfiguration >mirror: > helm: > repositories: > - name: podinfo > url: https://stefanprodan.github.io/podinfo > charts: > - name: podinfo > version: 5.0.0 > - name: sbo > url: https://redhat-developer.github.io/service-binding-operator-helm-chart/ > # local: > # - name: podinfo-local > # path: /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/local-test-clid-28/podinfo-5.0.0.tgz >``` > >Run all flows (mirrorToDisk, diskToMirror and mirrorToMirror) > >`mirrorToDisk` >``` >./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/clid-28-v2.yaml file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28 --v2 >``` > >`diskToMirror` >``` >./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/clid-28-v2.yaml --from file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28 docker://localhost:6000 --v2 >``` > >`mirrorToMirror` >``` >./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/clid-28-v2.yaml --workspace file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28 docker://localhost:6000 --v2 >``` > >## Expected Outcome >All flows should finish successfully and the images should be in the target registry as showed below: > >``` >curl http://localhost:6000/v2/_catalog | jq >{ > "repositories": [ > "redhat-developer/servicebinding-operator", > "stefanprodan/podinfo" > ] >} >``` > >``` >curl http://localhost:6000/v2/stefanprodan/podinfo/tags/list | jq >{ > "name": "stefanprodan/podinfo", > "tags": [ > "5.0.0" > ] >} >``` > >``` >curl http://localhost:6000/v2/redhat-developer/servicebinding-operator/tags/list | jq > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed >100 801 100 801 0 0 316k 0 --:--:-- --:--:-- --:--:-- 391k >{ > "name": "redhat-developer/servicebinding-operator", > "tags": [ > "sha256-16286ac84ddd521897d92472dae857a4c18479f255b725dfb683bc72df6e0865", > "sha256-30bf7f0f21024bb2e1e4db901b1f5e89ab56e0f3197a919d2bbb670f3fe5223a", > "sha256-67c2a2502f59fac1e7ded9ed19b59bbd4e50f5559a13978a87ecd2283b81e067", > "sha256-ac47f496fb7ecdcbc371f8c809fad2687ec0c35bbc8c522a7ab63b3e5ffd90ea", > "sha256-f79f6999a15534dbe56e658caf94fc4b7afb5ceeb7b49f32a60ead06fbd7c3fc", > "sha256-e4259939a496f292a31b5e57760196d63a8182b999164d93a446da48c4ea24eb", > "sha256-e01016cacae84dfb6eaf7a1022130e7d95e2a8489c38d4d46e4f734848e93849", > "sha256-de1881753e82c51b31e958fcf383cb35b0f70f6ec99d402d42243e595d00c6dd", > "sha256-69a95c6216ead931e01e4144ae8f4fb7ab35d1f68a14c18f6860a085ccb950f5", > "sha256-cc5aab01ddd3744510c480eb4f58b834936a833d36bec5c9c13fb40bbb06c663" > ] >} >``` > >IDMS and ITMS should be generated as showed below: > >``` >📄 Generating IDMS file... >2024/10/02 16:10:25 [INFO] : /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28/working-dir/cluster-resources/idms-oc-mirror.yaml file created >2024/10/02 16:10:25 [INFO] : 📄 Generating ITMS file... >2024/10/02 16:10:25 [INFO] : /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-28/working-dir/cluster-resources/itms-oc-mirror.yaml file created >``` > >`IDMS` >``` >--- >apiVersion: config.openshift.io/v1 >kind: ImageDigestMirrorSet >metadata: > name: idms-generic-0 >spec: > imageDigestMirrors: > - mirrors: > - localhost:6000/redhat-developer > source: quay.io/redhat-developer >status: {} >``` > >`ITMS` >``` >--- >apiVersion: config.openshift.io/v1 >kind: ImageTagMirrorSet >metadata: > name: itms-generic-0 >spec: > imageTagMirrors: > - mirrors: > - localhost:6000/stefanprodan > source: ghcr.io/stefanprodan >status: {} >``` > Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Foc-mirror). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
openshift-ci[bot] commented 1 month ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aguidirh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/openshift/oc-mirror/blob/main/OWNERS)~~ [aguidirh] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
aguidirh commented 1 month ago

/test images

aguidirh commented 1 month ago

/test unit

lmzuccarelli commented 1 month ago

@aguidirh I'm going through the code and then verifying using sonarqube

lmzuccarelli commented 1 month ago

@aguidirh Executed against sonarqube - code coverage is at 74.6% - but its no big deal, as overall we are still at 80% (which meets our agreed AC on code coverage)

Cognitive complexity is high , but honestly in all our logic the value has always been high. No other issues regarding Clean Code attributes - Consistency, Intentionality, Adaptability, Responsibility and under Sofware Quality - Security, Reliability, Maintainability no issues.

Overall looks good

I'm happy to give it a LGTM

lmzuccarelli commented 1 month ago

/lgtm

openshift-ci[bot] commented 1 month ago

@aguidirh: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).