karmab / kcli

Management tool for virtualization and kubernetes platforms
https://kcli.readthedocs.io/en/latest/
Apache License 2.0
496 stars 136 forks source link

disconnected registry upstream URLs #709

Open ElCoyote27 opened 3 weeks ago

ElCoyote27 commented 3 weeks ago

I have a working proxy registry located at: docker://registry.lasthome.solace.krynn:5000 It properly proxies for quay.io, registry.redhat.io, etc...

When I configure kcli to use my registry, like this:

disconnected_user: openshift
disconnected_password: OBFUSCATED

I see this being configured in the kcli install config:

imageContentSources:
- mirrors:
  - registry.lasthome.solace.krynn:5000/openshift/release
  source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
- mirrors:
  - registry.lasthome.solace.krynn:5000/openshift/release-images
  source: quay.io/ocp-release

As a result, the bootstrap node fails to come up fully and spits out this message

Aug 20 11:09:56 ocp4d-bootstrap release-image-download.sh[363447]: Error: initializing source docker://registry.lasthome.solace.krynn:5000/openshift/release-images:4.16.5-x86_64: reading manifest 4.16.5-x86_64 in registry.lasthome.solace.krynn:5000/openshift/release-images: manifest unknown

I believe this configuration is wrong and that the following configuration should be used instead:

- mirrors:
  - registry.lasthome.solace.krynn:5000/openshift-release-dev/ocp-v4.0-art-dev
  source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
- mirrors:
  - registry.lasthome.solace.krynn:5000/ocp-release
  source: quay.io/ocp-release

When I check the upstream of these repos, I see this:

$ docker login quay.io
Authenticating with existing credentials...
Login Succeeded

$ docker pull quay.io/openshift/release-images:4.16.5-x86_64
Error response from daemon: unauthorized: access to the requested resource is not authorized

$ docker pull quay.io/openshift-release-dev/ocp-release:4.16.5-x86_64

4.16.5-x86_64: Pulling from openshift-release-dev/ocp-release
Digest: sha256:ac78ebf77f95ab8ff52847ecd22592b545415e1ff6c7ff7f66bf81f158ae4f5e
Status: Downloaded newer image for quay.io/openshift-release-dev/ocp-release:4.16.5-x86_64
quay.io/openshift-release-dev/ocp-release:4.16.5-x86_64

To me, the first URL is wrong (even if present in some docs) whereas the older URL works...

With the following patch to kcli, I am able to deploy with my disconnected registry without issues:

diff -r -u 1/usr/lib/python3.6/site-packages/kvirt/cluster/openshift/__init__.py 2/usr/lib/python3.6/site-packages/kvirt/cluster/openshift/__init__.py
--- 1/usr/lib/python3.6/site-packages/kvirt/cluster/openshift/__init__.py       2024-08-16 20:54:49.920500082 +0200
+++ 2/usr/lib/python3.6/site-packages/kvirt/cluster/openshift/__init__.py       2024-08-16 20:43:03.029192666 +0200
@@ -150,7 +150,7 @@
         for extra_release in extra_releases:
             tag_and_arch = re.search(r":(.+)$", extra_release).group(1)
             synccmd = f"oc adm release mirror -a {pull_secret} --from={extra_release} "
-            synccmd += f"--to-release-image={disconnected_url}/openshift/release-images:{tag_and_arch} "
+            synccmd += f"--to-release-image={disconnected_url}/openshift-release-dev/ocp-release:{tag_and_arch} "
             synccmd += f"--to={disconnected_url}/openshift/release"
             pprint(f"Running {synccmd}")
             call(synccmd, shell=True)
@@ -978,7 +978,7 @@
         update_pull_secret(pull_secret, disconnected_url, disconnected_user, disconnected_password)
         data['ori_tag'] = tag
         if '/' not in str(tag):
-            tag = f'{disconnected_url}/openshift/release-images:{tag}-{arch}'
+            tag = f'{disconnected_url}/openshift-release-dev/ocp-release:{tag}-{arch}'
             os.environ['OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE'] = tag
         pprint(f"Setting OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE to {tag}")
         data['openshift_release_image'] = tag
diff -r -u 1/usr/lib/python3.6/site-packages/kvirt/cluster/openshift/install-config.yaml 2/usr/lib/python3.6/site-packages/kvirt/cluster/openshift/install-config.yaml
--- 1/usr/lib/python3.6/site-packages/kvirt/cluster/openshift/install-config.yaml       2024-08-16 20:54:49.923500125 +0200
+++ 2/usr/lib/python3.6/site-packages/kvirt/cluster/openshift/install-config.yaml       2024-08-16 19:46:05.904662601 +0200
@@ -127,10 +127,10 @@
   source: registry.build01.ci.openshift.org/{{ release }}/stable
 {% else %}
 - mirrors:
-  - {{ disconnected_url }}/openshift/release
+  - {{ disconnected_url }}/openshift-release-dev/ocp-v4.0-art-dev
   source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
 - mirrors:
-  - {{ disconnected_url }}/openshift/release-images
+  - {{ disconnected_url }}/openshift-release-dev/ocp-release
 {% if ':' in tag|string %}
   source: {{ openshift_release_image.split('/')[0] }}/{{ (tag|string).split(':')[0] }}
 {% elif 'registry.ci' in openshift_release_image %}
karmab commented 2 weeks ago

the mapping is done this way to match what's done when using oc-mirror, which has turnt into the standard way to mirror ocp content.