yrobla commented 3 years ago

Can you clarify a bit more about this? What's missing?

maheshd2 commented 3 years ago

Currently I see there are references in the code which requires access to git repos and also install many dependency packages. So in an environment where we do not have external network connectivity, the scripts will fail today. So we should support ztp scripts in an offline environment.

ijolliffe commented 3 years ago

A couple of thoughts. We did simulate an air gapped environment in our demo last year. Do you have some examples or a pointer to help us zero in to the areas you are seeing?

maheshd2 commented 3 years ago

@ijolliffe Please let us know if below links will be accessible in air gapped environment.

https://github.com/...

http://linux.dell.com/....

https://raw.githubusercontent.com/...

https://mirror.openshift.com/....

https://cloud.redhat.com/...

pmallan commented 3 years ago

@yrobla any update on this?

yrobla commented 3 years ago

Instructions on how to install OpenShift in a disconnected environment are covered on https://docs.openshift.com/container-platform/4.7/installing/install_config/installing-restricted-networks-preparations.html However, we haven't covered that bits into our playbooks, because it's focused mainly on cluster installation and control plane. Also, some helper script that is normally used can be seen here: https://github.com/karmab/kcli/blob/master/kvirt/openshift/disconnected/registry.sh

ijolliffe commented 3 years ago

The work here was intended for a demo originally, we would happily accept pull requests to expand on this work.

pmallan commented 3 years ago

@yrobla We have setup local quay server and could mirror this repository https://quay.io/repository/openshift-release-dev/ocp-release:4.6.*-x86_64. Do you have list of repositories to be mirrored to install ZTP cluster?

yrobla commented 3 years ago

Documentation is https://docs.openshift.com/container-platform/4.7/installing/install_config/installing-restricted-networks-preparations.html . You need to mirror the release as explained there. As a guidance you can use this helper script https://github.com/karmab/kcli/blob/master/kvirt/openshift/disconnected/registry.sh and fill with the right values for your version.

Rekha-Udabale commented 3 years ago

Hi @yrobla

Here are the steps we tried till now for the air gap environment: 1) Set up a quay registry on OCP cluster. 2) Mirrored the external repository quay.io/openshift-release-dev/ocp-release with tag 4.6.28-x86_64 3) So the mirror registry has image and can be pulled as: docker pull vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.28-x86_64

Now we are trying to deploy the cluster so that the images are being picked from the mirror registry that is setup.

Changes updated in inventory file:

# Parameters for a Restricted Network installation
use_mirror = True
mirror_certificate = "{{ lookup('file', './mirror_certificate.txt') }}"
mirror_registry = vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com

and pull secret

pull_secret={"auths":{"cloud.openshift.com":{"auth":"b3BlbnNoaWZ0LXJlbGVhc2UtZGV2K29jbV9hY2Nlc3NfZGI1MmY0MzQ4NTIzNDg5MTk2MWM2MjZlMjNmYTU2NDY6UjhHVVBJTTdJQ0ZXTFpZMk1GVEQ0TEMyQ0tLVVAwREFZMUNVWDRHQTdMOEkzWU9PREw3SFhDOUM1U01OOFhZQg==","email":"pmallana@in.ibm.com"},"vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com":{"auth":"dnJhbmFkbWluOnBhc3N3b3Jk","email":"pmallana@in.ibm.com"}}}

Updated the patch-install-config.j2

- mirrors:
    - {{ mirror_registry }}/rudabale/ocpmetal
    source: quay.io/ocpmetal

With the above changes we are getting the below error (the command was run manually for this run):

[core@sos-master-2 anchors]$ podman pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2
Trying to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2...
unauthorized: access to the requested resource is not authorized
Error: error pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2": unable to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: unable to pull image: Error initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: (Mirrors also failed: [vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4: manifest unknown: manifest unknown]): quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized

As we see in the above error snippet, the pull from quay.io fails because we had removed the entry from pull secret intentionally. We see that pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 is based on digest and not the image name. How is this digest decided and configured?

Also from this file - https://github.com/sonofspike/cluster_mgnt_roles/blob/master/create_cluster/templates/patch-install-config.j2 we understand that there are two repositories we need to mirror : {{ mirror_registry }}/ocp4/openshift4 and {{ mirror_registry }}/ocpmetal Please let us know if any other registry needs to be mirrored.

pmallan commented 3 years ago

@yrobla In Assisted Installer, which pull secret to be given when we want to work with local registry. I created pull secret with our local registry and provided in the AI. is that correct approach?.

pmallan commented 3 years ago

@yrobla please find attached installation logs. I see this error in the installation log file. As I mentioned in the slack, we need a clarity pull secret to be given in AI

sample44_72ea9b3e-77f2-4a18-8094-690b41a1ebe4.zip

May 20 15:56:24 mgmtacm-master-0 installer[8703]: time="2021-05-20T15:56:24Z" level=info msg="Extracting ignition to disk using quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 mcoImage" May 20 15:56:24 mgmtacm-master-0 installer[8703]: time="2021-05-20T15:56:24Z" level=info msg="Trying to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2...\n" May 20 15:56:24 mgmtacm-master-0 installer[8703]: time="2021-05-20T15:56:24Z" level=info msg="Installing Red Hat Enterprise Linux CoreOS 46.82.202012051820-0 (Ootpa) x86_64 (512-byte sectors)\n" May 20 15:56:25 mgmtacm-master-0 installer[8703]: time="2021-05-20T15:56:25Z" level=info msg=" unauthorized: access to the requested resource is not authorized\n" May 20 15:56:25 mgmtacm-master-0 installer[8703]: time="2021-05-20T15:56:25Z" level=info msg="Error: unable to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: unable to pull image: Error initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized\n" May 20 15:56:25 mgmtacm-master-0 installer[8703]: time="2021-05-20T15:56:25Z" level=info msg="failed executing nsenter [-t 1 -m -i -- podman run --net host --pid=host --volume /:/rootfs:rw --volume /usr/bin/rpm-ostree:/usr/bin/rpm-ostree --privileged --entrypoint /usr/bin/machine-config-daemon quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 start --node-name localhost --root-mount /rootfs --once-from /opt/install-dir/bootstrap.ign --skip-reboot], env vars [PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=xterm HOSTNAME=mgmtacm-master-0 container=podman HOME=/root], error exit status 125, waitStatus 125, Output \"Error: unable to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: unable to pull image: Error initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized\""

pmallan commented 3 years ago

Am not able to pull this image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2" using our quay id in order to put this in our local registry. Also do not see this image quay registry. The documentation or the script you shared above do not mention about this image.

pmallan commented 3 years ago

@yrobla If I provide pull secret like this in the AI then installation is going through. As you see below, the pull secret has entry for local registry, cloud.openshift.com, quay.io, registry.connect.redhat.com and registry.redhat.io. But this is not our requirement as you need connection to access to registry servers. We are looking to setup local registry that will have everything to install OCP cluster. Hence I had requested to share no of registries, repos and images to be mirrored locally. The documentation and script you shared helps in mirroring one repository openshift-release-dev /ocp-release. Correct me if am my understanding is not right

{"auths":{"172.29.231.249":{"auth":"dnJhbmFkbWluOnBhc3N3b3Jk","email":"pmallana@in.ibm.com"},"cloud.openshift.com":{"auth":"****","email":"pmallana@in.ibm.com"},"quay.io":{"auth":"****","email":"pmallana@in.ibm.com"},"registry.connect.redhat.com":{"auth":"**","email":"pmallana@in.ibm.com"},"registry.redhat.io":{"auth":"","email":"pmallana@in.ibm.com"}}}

pmallan commented 3 years ago

oc adm -a pull-secret.json release mirror --insecure --from=quay.io/openshift-release-dev/ocp-release:4.6.16-x86_64 --to=vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 --to-release-image=vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-x86_64 info: Mirroring 121 images to vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 ... vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ ocp4/openshift4 blobs: quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:66871bedd1262ac24abb0e19c0d43f6265c211203d4b44a46c1cbdb29dacbac8 1.36KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b24086fcc584e1fd52a0c2c0223a068ddcbbf79ef1b73e1c82a648524f8834cd 1.376KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:927653fd3e6be7280b25b39340f3166ea8f32898667f01800afc89bd99b39943 1.834KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b52c214df8e2c1d40f8a56300d2399b430c856f2af952d75fac5972889325463 2.562KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:0db75304394d9a864166173a5be3b72e44f305cd50a7e04ca337b74d340d253f 4.106KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4334b31ebd9579af383ab8d87469469aa8ab763a81006635733c8a592fb6986d 4.56KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:74b4e9f02d9efd4187d4d3965005f4b7435a1a3b780c676aa3ce45cdfa2af4a8 4.604KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b738c09c6dcc0b180e5789fb6ad38d16f41d312c9c5d6f08575e3044947cc85a 4.771KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:281800d028a252aaafdd5d1a827c9540839e95ad3ca8d32a6ac850e52af247ba 4.865KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:0f8754b67ebea6372d40e7099ab5af776fb0e566fb43f50c45fe116423780b45 4.871KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:579400e615b2e1b57f28948410ad023a71bdff0f28bd5c6d1f94a46d2aec11f1 4.899KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ecd239e3aee6a21dea30b0e60940754b6ddf9a606a2ea28b4050c42b6fa66f39 4.903KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:769866a8fe21694c7c97bf5a9b26949c85cca3da28844459138a74328e5c615e 4.907KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:cabea1643869c69c5e7411f3136dc66b2531eb16681e323a9fe6227880f3e0e1 4.966KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:cd282c2092ef84e7d3b73c25bed67aebbb006878fb61aa2ad484aee61df67cd9 4.986KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ecf1c9072f8dc84ca7b468e8ea25d7e734496272b9cacc6c13c2ee5c451f8c78 4.989KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2c080c9239f213ad65232584d0e8b291e04b34b7bd3afffd00fdd494a8f62354 5.071KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5848b747830e5bdccb95c8d230fdcb3fb01a3988f8298205ad8398c5bcdf0141 5.135KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:271e1560a4597318f07d74ff90cb2da709e9b941729046f05c107672df4348fb 5.15KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:07e316fa902b7e2897ab5ce853de4b88ae367d6e2831d13e0b7535725b824d91 5.154KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:cf12fcc57f199408226a99298133a44c3d0dab29bdd4e120355e8c5c0f81a145 5.168KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:07cb764bb54ad6a13fad477db11eb25312e00fbc351636a9ae4622634a059041 5.197KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:02ea92708f641b338b17f63f8e37179f8ca63d532939bd93afcfcf18100b46a8 5.218KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b1b14e9c6661765427a71a2a0940ca711397126ece3a6b9f2b3a35f4ecd04852 5.281KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2c460acefc24c76146a37d8f11ba3534ee556bb29d6229c1e1e741cd061b5c2e 5.347KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:0c45467511e66b9f31547eebf7630df04564d8ffc350c3b7e691399d11531da2 5.364KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:c27d9f58d5b5617a8b15a243ac7989a0028a122ec1e9397c68512e1dcb9defae 5.368KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:3b3e71814060928d9e984439384021e1688d4753addc5b6f87424d8723877316 5.385KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:3ef62c2cbbf8979b3642c0578176cbef0289a36e4f1d079073444003123c8b1e 5.387KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:64a7bbdcdee2776744ea1a91e61dcda36aefbf644ad0918b487e5bb896394e1d 5.407KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2d64b01814c556cd749175fb73f1b723a72fa7f205cee43bb8e3841e4257f055 5.41KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:6953e57d76f8557febca80fddb23eff810bdef41d4b85b74d2edf43a09a1d725 5.412KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:712747ead7d584c7dac029be303d338bc882832b93d302eb63f076450f156a4c 5.415KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:278d8f4066a5459a044de406ba3d7e3c62025c05260755616928c8c797be3125 5.417KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:8b298a9473c6a1f2f679af113603c91e42972a98f0da04c7617212d6ba42b964 5.42KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:6b00ce2f68cc420ba19574702e18f91edd2c1986b23127c8bbf0d2daa80cc637 5.437KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:8bb7bde21188d232f2018e609c486d84641862cdeaee0650d861f33f08b99e12 5.447KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4dcaff3905919b843525c52056a980fab6176746aee752c711e7896deb76a592 5.473KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5ed187ae8f4f0b157930cdc9c169cc64b17790b1b54079e74b7fbd921d8f5176 5.479KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:602af6904e22603206444f200f99fbe2300e9d2435d048dfd3ef6ea020525ea1 5.479KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:67caabb301339ab9b08766a3a7639ef048b10e3a856c99d56b250929f827177c 5.481KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:94581e358a152a9953911ce6ed8247cf8c8ad0cc45c4a9a1232a4640fa04510f 5.482KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:995d948ff019b282a68674fd7198677dd7291681c7b728fc4f84963946d7cb3a 5.485KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:8f8899966bfbf27bab37cee477d3074a47b20fa8435cf0eb06db4234bac43d1f 5.5KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:1d17a8477fb390092a200cc76004fe0aa730641d7a38e0bfd587433413daf5e3 5.505KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:1b5ec8c1856408051dec96adb631cf6df1a3154ebc4ab6b17fd6881bb6d9c99d 5.509KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f312ba53659cd7d4ce11aaeb9903831af6d533f11bbe6ca049202b912759ae3d 5.517KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:6cc03871a6c7bd887497fba6452871a6f431608019fb2d28ca98e00a51788f86 5.518KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:606763e586d644f569c45aaffbab5c9659ff9674e9dab47350a8b28cf3334dc5 5.519KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:e9645806f43f2b13cbf54e2d6caaf130398b4cbe560553ff613a9d180eb674af 5.528KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ed64d0b56df95a75dbff0bba4fee4cb54185846f5abaa79f1c00c49a46553b64 5.529KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:34bee8e60863a1e1cc00b673e5cff5c78ede0f52e2e181d4a941984e861d6f7c 5.534KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:fc1f2467652812bee47e2099761e8f8b1946513cda2b4943b87e3adfe3e564d0 5.534KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b24f036ba2ed3d91f7cadb93910c746d81c19cdb4912a31d49ed56bc692369eb 5.543KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b5eea13ef08b66f1cfe7df8b9708ac2685b82df0c491b709bd7d408d506aa6fe 5.545KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5694f8be0e785cd0ab510b265f15b3c58b5f530b9c47a7046fea7044a7830a91 5.556KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:9d9da0150298450b49d7744558ccb7c0b15cbb386fcfd04f27da9e2786a8d081 5.556KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:0f4abcb986e497e950f7801dbcb8a6079c1256522564791b5eea64bb64392e6c 5.559KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:6c1d995a225790b2658e40839af6ed3133aaf3b97db86f15f73051ed4e772431 5.559KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:cc2aa2b60e0268cdb6c05b5dbb851a2e28f7893795a406264b18c0a53527fffc 5.559KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:66efb1edd602132e9618eed57008c77f5ecb0c66260c8b138cde2098d490ca7e 5.561KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:7eff493dda21287287840aec44af34d27c7719cef292deea23642d641c364fe0 5.562KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:9dc7c9c9c60a235665074dbbd5506d23040e7fd4e1128cea60d726aec220f841 5.571KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:76f87939fd60d9c40648eedd7322d83c26f298180417e897c995813fcfefc3b8 5.573KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:da154c237dcae4e3fa5e1f8074e4dee26c64ffb7fa37edf47f95371ad1877f50 5.573KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:3353636ca20e879b05711841f68f5e745bcc96e601f10768dda09c3877aa9cc9 5.574KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:1326be0b0bef1dccc089b7202989d24b666071785cb8107dea248265a341f0d7 5.579KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:0e6f371a89b2aba0c2d0d9898a8ffb2c47e83a40ae0e70896f8c7f1bc5d9fb77 5.582KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4621afc8d94bd942b8fbe8490621786163fdb3b945662330dc021595e6c90de1 5.582KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d9f2d105d555fe9890e3e4582fe53351fdcddb9d4762da9476e463098674f79a 5.582KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:63ed2496f29ac691b04f22322b6289201194cf8fc6fb2ec0c14fa84c6d4bad4b 5.583KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d5b0b426896b1d230bcdbf2a75b0125154786e0b3a2ee5c921f5396b821aa8d7 5.592KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:e74946514e798f25107b624454f5455149f32fdf8b4bdd1d2dd0e09e34ba034e 5.592KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:50513f0846173b546c4d498f621b769e7109c6bf2ed0bc14849b5c258bbb112b 5.593KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2c6f8c1cb81478d327996dc0a82ee876bac08cbaeb0f7dad1d2fe7f1aaae56ec 5.598KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4f542348d6bc9b9117e5ad3ec604cd7c40cfaa187dc2b4ab1aa07521fb3cb959 5.601KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:c0d49a9035045ff42911322104e6f36b2106d1bda7775ee1182621b02a0f597c 5.605KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4d43a7cbd40e78c6c4da458d8c0cdde15da1f98e6510dea677381e654818b3db 5.607KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:0f5f261d955440e15e55ae5febd5aa8c68584b9ba4dca1af5eba3dfec51e483c 5.609KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ae586c97b490eccca38bafd9a5ba13b0c4e2641d1e934c08c5ead0218632deb0 5.609KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:05fb5fcc2d87d7b15c16c245a32833b778aa1fd379170b902398be49fd5819d3 5.61KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:40093321c0a40da571e606820414ab47fef636cee85478a501e10a497cbdaca9 5.619KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b7c72589ca36f52d32e433260a4fc80b2438169a7c58b58b203a1cd429c0048f 5.625KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:a686d47b0e02d9a2349b7d85f9ec488463bfc0deb14e0231b09482e632c62a5d 5.627KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:25aef82205de9de8463d71f5a1518bdf2fc1948e7dad2f3f7fc157b5fbb93459 5.637KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:0f94963a899587730af482e00970f325ceb31426e4d43dc78f511c441de1f474 5.639KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:75fbc7669a0a98c4c7968d1f91a8c81c061a4cdd1758696e05b1dadaf291072c 5.641KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:47fc213b0a6939c36bd7aba28e4a848345a27d5d1fd70266a68cd1a45c3e11ce 5.643KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:080de29cde8ea8c73a8099aee805c1a18d44eb5ddca2abf72cf6a6b0e90ed221 5.644KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:edac2fcfcb7199073d01e3e7a14bbff63d91c3bffc9b8316e7abe86783e2159d 5.646KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:c369a7f771891820d545b007e8d19e5bff3ab7b76e5807f70a9ccd2357c4d64b 5.652KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f1825b4378b0a906d7642fbfac183e32389633d3a95820e8b1d86f5fb2de8d46 5.653KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4cc47cca2c102c1a88bcb140221a903afda3a07abb4383781d451a6bebd34d0d 5.665KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:6eb0463a7a61135dc1ff92fde56118c1f26d250c85939172e9e3fa54cd809470 5.666KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:525613130af89b0c195da91497c0e37787d98ae77636c61c181ea8d56ddf1ed7 5.671KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:6a3af1f8623dfd0d95f74d812426c6fce89320ccf37d1775e437ba5f0cff46cc 5.671KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:7ddb38781cc890a3afe60b9f338ddb4c5fdd91790d2b2e5f3115d1bee0940ca7 5.671KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:1a18f3e05a59d07b752ce93223fe7823122338c30b85400ba883c391c09ce63d 5.676KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:3948adf11d948afbb8d53b41d0ed538f57cf892f71ee44e2f96d90b60efc54d4 5.677KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d9da1593346b8d4bd5b65374a39b6d92f8e1909012bdb7558e89e826958fcbed 5.681KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:0e53d024fde63c1c27ea70f65f1f4067f1a62564ad07cc9bfa11151ce18961be 5.683KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:7400ffc3001159cd52194a8d3fc3a653726815f0191a2a421781a051fbb089c9 5.687KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:19414e19ac8ed5e880cfffecea3868ca4f976fc7bcd8550c7e401e9289eb2946 5.688KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:26380dd3882d93d5874de847d705f4d96a49e364a7ed1e893219aa0970418341 5.691KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ecf16e2ff0fadd56283cce484a8496ba39f52ac9d6c355c37596b2e02ab76f62 5.691KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:24aed4a92cd4bd64d1ce83b605ecd9e0ca582d839b79a9b9e0d692eae8470158 5.699KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f3cad32a0b23dbeeeffefdb1d7ff317e3c124c9022476cbbb0e692153dc0d93b 5.711KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:50f220b3f8ae2536b2a599856f9f06211e1f52c3dcec65f767ee7687b30c45a1 5.712KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d70ba80e423a507e3951736f9512feeb32f7d71ee314d890b8c6c41f4e457dce 5.735KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:7d93dda00fa07057b38bf33781d969d623fd70c598520a44f09d34a16aed6870 5.749KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d9b2926714695ad0b4ccf559ed627f702935cf06bf4d9d76feee45dcfcd8d7f9 5.755KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5fdc8e2395efe1ef47f065c9d7fda67d18a075004294e9b4ed249732b9d5d674 5.756KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:98357a733195beea7a30b1a0c85e5eb40fad2a6090ce573fdd3d013e5e926d90 5.798KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:bf825ed2e386e74ac8199792c8d330da68136b6bb1414711eda161ea5bb5338e 5.814KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4baff4da9ae54a9b1664aa7641d876054f64931e6d084572c024533d72c2a61a 5.829KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:866e7bd9fa87cf4717cfb02df6b73ddb6f6c6d31125b1e63ce9313feefcbb2ea 5.84KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:fab5511c70dfb4796d19fbfec29b5720dd1f72930937779ea4478f14db4f9641 5.897KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:9a5b10ced65372a9a59167a7c88cb1dbb76a3eb25294dc8d061a33788742fe05 5.952KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:a60f35854e1bc554ff254dc48d9fdfe43686d60f7b754e7d8f01702db57b804c 5.972KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:45db16cc68c03ddc898cf1cb9f8fda035cd33706d14110564152f674d83e1a61 6.117KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:32b894f38c2b061d80c1317da3e043d02386fe0042b179ba3e51fdfbea2544cb 6.156KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4e5d81ba29f1a398351c8cb117b4b2aa1caa1a671d87c1853982254e54355648 6.344KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:507cf965597b1e5f136383a597045f1e3819db0787e9d78a51a3291a47ff6c77 6.761KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:1051b62b6ba385300e3466b3a72919419f502876a4cccb74772a54b25ac78d2a 6.842KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:61ee7eaf1877a22133b94f2229d9be48ced050c5d483e56ff30a71e0b801fd72 7.898KiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:36468b62fcd3c6c7bc0793eae7a13ac1d2a9757f0009673911be6610eafe3075 1.157MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:06f855db5119587360772750b49d596c59e74160fcb6c609e69d846ae604f348 2.837MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:28786b00e405505def8342ec59d4063cad2a477b566c33106c4d81f81d932aef 3.427MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:622b8bb95c1476c2d692f6cb44dceab43d18ab3896d50fec375ca74dd162cad4 4.962MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5549446e3c52e9c866db0f259f491f22e9a85e823e47afe2eb2f31b9e0dece4b 5.086MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:658ea0558e8fc93b1d761e980b0daa6c97a213d9c648e058ec37c47f987783b1 5.127MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4061672a2b58d2c95a79b774938d9ed40e490491d6a3c995c746e43ee44b373b 5.522MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:3e4f6e27d6312a7306deb34d2b21fa9263a0686bd909f1984f939e80a3bc003c 5.918MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ce3c51b51603a30638fffb0f4fc56961952bfb4b4d9a096daf928049cddffa34 6.535MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b8054f1b540e41b87156847c78212beac889e9a11d9e98441cc95a286b8b7d92 7.404MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ca570ef983da226c8117457cfd8df66483cf39e9b0be2a76ca32499e0f840f31 7.995MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:90fdbc5e6d8e3313a268129c4f170e8e432834601b36f51ee0709b84dab634d4 8.134MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:fbf81b9a2d44e52bc19fe30813f3c31a701d405ccfd53724cd89afef1c61d2f6 8.335MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:198df30fb540a39ff3d54610b33886dfbb76fb5d12941e06ff312d2d8b2baa4f 8.602MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2b650abd7d8225d729a86f0fd81f5512514cdd6764fb8b0dce1b8902567322ee 9.027MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:bb68e8edd5efd73e622c6c623356c6aebf296121387195f74d08a65965f02bbe 9.153MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b6dc2de9c78d1bd8b8c7668769479e71cdba6b6432b4be41e68449e92c1812ac 10.5MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2a882f4331bbc7ab364c4c8788a8181a26d9499b0a8cd3ce1c512b37dd52a06e 10.58MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:a4428ac7e7770b7d06099cb24139ee8870dd811e8edd29b81d6ecf562ef3b18a 10.62MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ab7046598ec421a9f6fc383dd1e22aecee86c7e8beeab85662035072d5ee819d 10.86MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:54ad16f053f8c29152fca1353db0b23bda423dafb58704ac7e5d145e2c2b8552 12.43MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:aa7fc101f2b03cd851e7150db8de40d4ea3d7ce46f74dd601c3dcac04961e8de 12.76MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:c515503b8e2819a6e729ba73ca3caf6a36a41d56dfe237de1b6e4229a3706a1a 12.93MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:db0d4a0292552c80250c4e8141b14bd059baa54c43c0cdec37042dae69714ba4 13.58MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2a34810165844b67388e08a8887b4c26e35eb6b5e59e9d78dea80f432a13c555 13.7MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:fa2e341b8efcf6a3d3c119baf2c02cbc19aec128929278b64cc9ae1addc8b44b 13.7MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d1130823979ed7f8cc1a1dbb1af51d58777556bac9c0b41f487f5a60783e60c1 13.94MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d987afbc5509d81499d81e77c483c95bb38e10ff2e5797a04df4f839310c8bea 14.08MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:987721fc9e66b1f9bbdf1d93950f03c5acbbe759bd5d5b35a540fa6f71298064 14.2MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f8411f1ef5b9aba2789932ecb6a86a981cd5411aae951940733bf8b8c405fc8d 14.26MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:78c5794215329e13b370e572b835f3856d1c80782997e5d794ed6f400b8cc9d0 14.26MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f937b750e970a9573caa4f6bcfa21a897581cd7e210ee2032dc76b238bb6acce 14.29MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:301430eddadca205bb3935e39b9bea081ede0ec94f60eab100055b03d28f885d 14.34MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d489fd3245f8a64076e4272d393ae36e83dfd6fa882abda580c41dc456c74444 14.45MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:64c84472ff54efa0723bd2c8dd337d9bc5637cc2be9c58457dea6119022ba23f 14.53MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:30b2af4f3f04deb220e8af1a727ffd52260bff4f3280ae12632a95cd9b3e8a59 14.71MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:03242baf3a888b0d9c0d22f3220be8b83e2e3476113c1c77f90fa0271fc8843e 14.85MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:bd7a6a63db8b0577c6804b9c899de40cfa58f3405628c7a442de9ed073abca2c 15.06MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:9707c00490a94cd0480a598759e1b7b3972d623808aba696c71f99074c2837b6 15.26MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:18e9172ad4d5dbe29de81e57f12999f337091dc3cd5cd9977d1be7d2b2af0b32 15.28MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:3fa171eabb0e3e053d34e2ffc5a8fca873296f9c031d2fae187bf87c22401d8f 15.35MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:13ecc66841369ef97526de74a483933d92feafbc5610a8de43f1425de2bfd6b2 15.36MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:c83ff2e8ea1136235688c0f5e0e9d9d8e5703439783707f9d3e3643688b04f59 15.43MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ec51190d499c00c0031569c0e2d4b5de3985317a6eb07c34b3cc09059010c0e9 16.02MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:bf82e58856a3f282ca30f5d3b13a7597292d18dc8193940005c4fc483481d3fb 16.1MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:fdfecce9efa3b64c89e4af5e774f5c6905c3ee58237c4bde63b74957eeafd283 17.12MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f34fc43a4dac72e83211c8af863211b717a196440ad377a154ff9868f16565ae 17.24MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:71ab386e206d48a44c314df009629d597484ec4f367bb4d5aef20f194a01cb7c 17.4MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5a8f76dad18a9fdc16ac6a4f430ce92a26ba4ab94800c1d27d67901f3e044052 17.66MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:e15714bf0b39f1ae49eff9902d3065554ed1e9494860f2df2a116aab0091ff22 17.88MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2cb776079e5183b0cf8d23efe7d9a4e007e6a915d8fa8bf399b7c627b39fd4b6 17.91MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:6ba8ea8de6a2464c29827e4bbb7a08223c9295bfc7b9fc2d5969a8d5a4ba50e0 17.94MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:44a89d0c0e0a0953ca915dd70bd714f612fd5a6e4e1116d306eec61a1393eabf 18.62MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:cb58464b68e86cd1b59a21c093c8769fce34ae42554de001eb5ff99350696e87 18.83MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b16b5043391c7b6db273a2eb06ebc2e99257089e52f30c7fa2711033d97ad668 19.08MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:c90142f6e9b705dbf579fe16d65d5be6f5baedc8742de25b40ed8916d805590a 19.35MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:19cd21687bde9883b3efe4b17bfe296829fb44a5b9dce0c3119ffcd31394f58f 19.49MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f936adfd6ff7708efa6256445856ec15b08a52b7b3814aa291bbf33877cb0d86 19.62MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:a498b216fe63860e105f99ded9fa2fb99dc471f03f07c477cff0f178ea9f123c 19.92MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b722bf8bea9f0249f2fe93647db16701ca78bf9d33c1e98c599741d42977585a 20.74MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:3491a0460b37df32f4ce28919349f59efdaa07320c8f550f71d1de1ab84be7c6 20.87MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b0ef12868506f33beb04ec505e179fc54664595be5cf8421ffb4d28753c4e290 21.59MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f93c76b49a2eea09f023f71d2c27b626fd98b7b6f112afba8920a5e2138fd85f 21.71MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:641085b680dfde549b8709396d8d71d1afa6003056de98628ad4f358400c5197 22.29MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ecb87e445723ff7a1f695b2c93100b907ad0be34649f25c326c5c94533288b8e 22.87MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:52ec59fa0941954218b1f8877c44785790e3f3991afd738a359e2fa68d23a4e3 23.2MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:e23e2a16c5547563c88b3247461c0a05a59badad77d29ed47b54ed16840c48d6 23.23MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:3b66af967542f8d2e5c293554421e9c16abe78f86148a92f730bf55131ec83f2 23.45MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5142b5c90cb446401eaafbc7a5d8c2270a201c78020667665490b0e37a5e292b 24.23MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:b06028a95508df2495b10b14f631f7376bec91e7e3928c8d4730cff7c43c75ab 25.28MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:db3638342a1cc20d72b3e0a4ae3e0bbdf57e4d553281514197fe2d3639582099 26.59MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:bfc5b1853a0d337329002518f45b36300aee5ae8c7ca1a0b3dbb2f807cb62b48 27MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:0b12a65b7b9ee8ce89adb68c38120011dedefc0d0f5301f8d19310077bb883c7 27.05MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5bed23e75deed901fecb501aee9337b35f4fd5d5862f2c1387bdb770a012410e 27.18MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:41cf0e9a4669634d918026d5e7f9a198ec04084ff17d9e6003ca888baf9807f1 27.95MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:fdd39391986bf72d76f4a04aac2da35dc4c28ca4ca189b87b4382ac2b8a3f47d 28.08MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f4f4cd736d3eed339a3a768f6dacb9be3b5a4f04feee89b45281e76530fb426d 28.41MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:491c4e0b3ff186a5add54eb3d215ed43caec32b3ad750a51d2ce70dea7611d29 28.93MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:45f851bde7e1f1a9267bb6bb4a9a08d9117c8455a3ac1e94c8096b40e170e380 29.14MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:6c1171a658062d861d044d5a65677b3036803af79895ea795f01dba448c32e06 30.97MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2fd83570867e7782acf1933de4d0ef9e8e5211c22d7851cc0c025f56b30ecb75 32.48MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:e78ab868aa58d652199ceee490cabf1a75946309dac64e6e7fe7e64fe885f93d 32.7MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:8027498bb7351c31f1da80cdfa3dfda7ae1302f61c642c4882a49850975fcffa 32.74MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4822868ec24440f79e687eb033eb354ea7c362b00ffffc667d46f1fed2f7c59a 35.38MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:89b397c96bf4c337f1a6aa1f06923e03f5c5694344a75a4ec9c41edf83b1f320 35.51MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d845d5003cee057dd762e1626668675e10856f671f280ad9d3cc3d377ac470cb 36.52MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:a2510c607cba00019ab4485fbbe521350ac2fbb0cf919794d67a5e3bcb925a7b 36.77MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:022d193218d77ac322a1a77501a86982be712612dc9286250ef60d12e368ed2b 36.93MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:bf7b90eec382c2a01da77860a6bfefd81409279590bc94f226ec063a379ee4aa 37.85MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:310c83d07e46ee797d4e086c894d2215f00df7915a2ad2a4bd28960dfbfebbbc 39.18MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:caf27dc5812fd1b7fb8242ff15095f1c6b8fc102d115f0af95bd5fd375aa8b08 41.64MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:dc39e49ccb938880884668b1636945a4f73697a603995993d1458d388f2097a1 43.06MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d373fbe4ef281aed903d66cb6888422d49a555703daad17be45badaa95513fcc 44.12MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:71cb781fe003a4244c559239d5569fad64c82e0fb96c6e0bdf54bbd41cf6c27f 44.56MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:cf76770be7dd07268e8bb8bd85dc315bba34d496b2587587a81287527154c9df 52.61MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:9714581c5d74ab1090633c3c047dfc08beb750e568d0faaaa6d53c127f5ccc4d 60.85MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f895238caed4adeab5c96b4c6693cef6e8e566e45858d73adfd64d2f5fd10253 70.45MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:6f406e2413170ad6cc4597a74c0c47d876370aaa65c24070f8fbcc35cb4f7ed3 74.27MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:95efbc6c0025e6746c9c573fafb81c4af15895e756b1d06a8d0092ebdd7bf10f 74.96MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d2cefff4288f0e4c8f119f80ffb3132b886c841504cc84b8a74a8cc20b58e700 75.43MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:747dc137594d496704ece95c709d3e06c2afc5a67677f061c7fdc9028c0764d8 75.77MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:2755828e3e9f023713dbba02734f4fbd06bfa9edaf979514eebc9d07fc5dc5c3 81.65MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5a38303e3fff29c38288480ce733cc71d2005b0844a1dbf53fe243403d4c71ef 81.7MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:493cac71a7b80928513997f8c69681618b1b0116ac5ee95c5e4eba0b932d0c86 86.79MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:fe0f36c58742b57ca7660fec7c338f2f5d1c4ac768dca0300aef4ed3c8f898b8 92MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:f02dfcd805b96ef833ff8bd5464a5185f4a7b78e2905b304d399b4776e214797 94.94MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:e5ceb5aa87feeac9c0c67363c31d2fd2033d789db1014b9d86796c88dc47bda9 97.16MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:7e10e22d9cfac52cf531a5726c3b69559c3a730d8abe5696970eae06a5d6a075 103.1MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:c91c00ff34b7bbaeec641bee65ba246b0cd1856756ac010e9acf7615c3bab154 112.8MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:e47b394b2dc87ee5b91eaa257aa8e6c3aa80e238da5aa1920333d64068d1d88c 117.7MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:bb3369a5a0139a29dfca06cb3d881796ffa2aef2eeee8266383d93372da8e45b 119.3MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:a972e3f4c83ab0ab8de9473c498b0d04125ce22874d8812128b3d8c83eff968c 121.2MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:23cd0dd42cfa30801cd1b09b7ab6903a056612b14ae9111a8acf000a70425968 127.7MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:07911f4a50fa95f17dece4e4b6e5ca137edfd5cc6b8a734884b2744f1b9f951b 145MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:4151cbe3a62535e862a9f25114a0ea2edb4b8537d898ba8d52f67d5bfe62812f 145.2MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:19cddb0c9b275d3549292b78631e0c658cd6a75633cfff023de97f04ee4dfa3a 149.6MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:7d010bb4749572e44924b52cc66faee554b720f3d500d0f44a1963e4e7ad2664 190.1MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:d59ef1c4d9e1d38ed9bc3c1cf1f66724f099e125f841ca6bfcdf78d65fc88f08 211.7MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:ef8d5a3f24626311ccaed51b09ec0ace59a8d3220d715e4393d10ae7eef5cf4c 217.1MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:5297e07497ce1d2a1a79d7359fcd03a5ca47f5f7ba4f6c8a5479518f51f7875b 429.8MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:e129daff942e5eaaba6887c9ca1621e62b715158a684162be7f07589e0cf2961 477.5MiB quay.io/openshift-release-dev/ocp-v4.0-art-dev sha256:63abd44e70e49ffb7bab7df860d41e07f5ae408f0dcc68938d86f862d0783623 869.2MiB manifests: sha256:002441e0c2d60adca78f0b063dfc7594503735c0c913105588eb5f1393fd1b6a -> 4.6.16-console sha256:01548ee4f07a6cd9167849a2c2c95e0a64793579856ea053c06dcf1183b6c74f -> 4.6.16-ironic-hardware-inventory-recorder sha256:09e3228d294d07940efebc06301179cf376f50ca16efe21ba0e343ab12ffa131 -> 4.6.16-machine-os-content sha256:11c7c2604386ab6c5d7a5e5e7579abc20a51c57755faa1d995528f5096a1af27 -> 4.6.16-deployer sha256:13640217da2ae26debbd19c124adaee503a5ab077d3950450fccd76dfe0941b5 -> 4.6.16-cluster-autoscaler-operator sha256:15a7862dc4854599ee29c0541f49b47d21d24ca1a3ce0fd11d629d8e48ae035c -> 4.6.16-cluster-policy-controller sha256:171385f90bc48fc195d0409dc8f69e770baf51bb1cce6747df8f10596d0462cb -> 4.6.16-operator-marketplace sha256:18c9d24bdd566275c120a978835af9a14d66052352391113650982e47ac70387 -> 4.6.16-csi-driver-manila-operator sha256:1b2a3b09b8d6851c10cbac36362043a2446acebbe9f5d45b3c800cd49c3679d3 -> 4.6.16-ironic-machine-os-downloader sha256:1ca8bc8e75c39d96959565dac7690172f6caf3468c83123589874f6dd9ebae5a -> 4.6.16-prometheus-config-reloader sha256:1d21de46a169dda2c8535058f214fb7d910f65d765543658fc56c55103d8b250 -> 4.6.16-installer-artifacts sha256:1dde6c31e71824d17329beaac7c87ff5c5b1d7c1f0a272ec858fec1812e775c6 -> 4.6.16-cluster-version-operator sha256:1fc73f923f35676b887dcd25500a5ab24aa306f87ad885a546ae0235203c14bb -> 4.6.16-grafana sha256:21d08b20bc5fba9f93ac98a2e81d933e3b0207d002bba0966e534625efcb3e09 -> 4.6.16-cli sha256:21d669ab48ea20f11c540cc030fde3f09426c1edbcbf573c8b44e0c72bcd873a -> 4.6.16-ovirt-csi-driver-operator sha256:23d436573130137206e222a072e20d4769bf92f32a6a93772e1074a290c2ddad -> 4.6.16-cluster-ingress-operator sha256:2401d0e248e71a21a3db7a9e217130de949ee9c7ebc987af518537dfb68dcaec -> 4.6.16-machine-api-operator sha256:2a7d2cd3ae6e1d2e0bab3682f86f916ca93d93aaedb04cff478a72063557f090 -> 4.6.16-operator-lifecycle-manager sha256:2b5ee448395420861a3e80bce64780e809fd500b061bd40175f33708055aa985 -> 4.6.16-multus-admission-controller sha256:2c71b04a8b2bc5c9b97ff160cf8a1814c41945211e1484ce7a4a83c5cbc06769 -> 4.6.16-oauth-proxy sha256:2e6b62f9983aa08086e959694e3567bc269e42608d7c0de58ba571b10973e500 -> 4.6.16-k8s-prometheus-adapter sha256:3496c77d7281fd4854732c9b8a0eb09283a162ce0a4b8a3c96ba51184049f0be -> 4.6.16-haproxy-router sha256:3899b8fba1ab11d5cbff5581ca3bc7c06ba47eece58588cd53f93ead60e8ea4c -> 4.6.16-baremetal-runtimecfg sha256:395bb85d2a03330089eedc10d6c0f83137eb7996cc842a94ae16ba49b365eb6d -> 4.6.16-multus-cni sha256:3a776866d39bcd264d389b75616c9dc32e6af7ec740192d252600fc1da63a488 -> 4.6.16-prometheus-operator sha256:3b8dc85390742e2a949639a410188a9202f9fffce8a32d8131fe428e79b89a70 -> 4.6.16-kuryr-controller sha256:3e855ad88f46ad1b7f56c312f078ca6adaba623c5d4b360143f9f82d2f349741 -> 4.6.16-x86_64 sha256:3fcddac914fca3cdf0884aff6a21dc45ac83e1a5b94fc419b5ef54ff3647e12f -> 4.6.16-openshift-apiserver sha256:43a56fd67737795384bdd7b8da9763780c6aec6b0fcf39d78c25378e35ff0ce8 -> 4.6.16-csi-node-driver-registrar sha256:4460e0ce7e047b86b9ad63d56cf0b953b419b47bd851b07e7ce9aa3deac64d86 -> 4.6.16-mdns-publisher sha256:4491eb6118d48f2c87c313464f6cccb8c290fb855df4446f2d3c37119f5b102e -> 4.6.16-cluster-autoscaler sha256:44cd3de006e68c454292db866b16417bd7dd77d0becfd2b30593b698520b08b5 -> 4.6.16-cluster-bootstrap sha256:44e20aa95a63f06449264a1ac0165c5311dab18840c82e6e4533d6bd38670772 -> 4.6.16-prometheus-node-exporter sha256:45f89b0f1fc62eded3ed3482f5574d41f374ea2e463b0c587f32dccb6f2f127f -> 4.6.16-cluster-network-operator sha256:48c2c43befaab410dcd1b296228150b8a1777c6ad0bdcd194e0a9087be3a2a6e -> 4.6.16-jenkins sha256:4cff4fb61970d98c666f3e1a6fb4d0c6a25e557c1344b281e08e605235250ded -> 4.6.16-oauth-server sha256:4d6d9e55b48edaeba59091abbcee46162b9153d44b9aac30c98ac94c6bd085cf -> 4.6.16-coredns sha256:505351039d9b93edade905f688e25d827e5d36b3e81878b927a47b7fcd7a1df2 -> 4.6.16-openstack-machine-controllers sha256:5696970cb8306c3028d6160a8ff02b03588c3323154473f4fd374900c918909d -> 4.6.16-oauth-apiserver sha256:599957b272e4012557e5792ad5a521fc53b4c7ec277ed888e8cbda40e5f488fd -> 4.6.16-docker-builder sha256:59ea6df65ba624a0421e050777be4a229d2b66c5ee741adb38d4b1e2eb939035 -> 4.6.16-prometheus sha256:5aad4db5d16c0c85ee1972184e0cb9e7ade4ae9f8a34044a9a306c684aa84e00 -> 4.6.16-docker-registry sha256:5dc1aaa71109b1cfb8e77402ae1db78ac1b08ec1b136e55ea8c4ba90cedca320 -> 4.6.16-cluster-samples-operator sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 -> 4.6.16-machine-config-operator sha256:60a313ac29046e0e0c31679a4b2187f94ee389e647b6b98bd72bc62b32632ac9 -> 4.6.16-cluster-etcd-operator sha256:624f99bcdc9fa53c0b48c7be6fb54056570201cbb5b5bad8b4029055235e037f -> 4.6.16-local-storage-static-provisioner sha256:63b7992c8e200d7669bc5b81bcc1f4778fde9c65aca6ae026ed1a32bcfaa18e1 -> 4.6.16-telemeter sha256:65990aba7cc00e680b34fcd83e9cc9704d45cd534f8c85232934f560f24281a0 -> 4.6.16-prometheus-alertmanager sha256:65d1479d2c780938afc97aa642c0d40299893c9616c2763247041bc30ec0c364 -> 4.6.16-cluster-config-operator sha256:690849c8f2824839e8730f0aa909e26f5ffff9f92355ef9e63b51441b9d8a2b4 -> 4.6.16-csi-external-provisioner sha256:6eb536df1ab94d857c363da104c5d47a37c11a0b8edd2a0e2b04f8437996ae75 -> 4.6.16-jenkins-agent-maven sha256:6edb934ac430fcd1a2da4f5ba4ed2a2f5e5017e7ba970c1fcf44b827f0a05776 -> 4.6.16-thanos sha256:6f5ae49f0d437e0f9ad25d5342576f8a7b052f9d676d926c35c92f2d79034cc4 -> 4.6.16-kuryr-cni sha256:6f6b06d31391d43e69c30d56a95677b32730da48160562e49911d1c1b14b034b -> 4.6.16-container-networking-plugins sha256:70b792bd18095d067b20dbf79309e53379f9a039ef4dc8b8e065bb209a8f0a33 -> 4.6.16-openshift-controller-manager sha256:71728988bb976b12fc435ef06b02e4822a89632f6136d96943f81009869d114b -> 4.6.16-kube-rbac-proxy sha256:71e1ed11e92fa647249ed4efbd53679ca459466761efc926baf7d2e4583e4562 -> 4.6.16-configmap-reloader sha256:731c0bf07c47678465a2f9b344f7f94de4f6990538411ced28f8f07fc0bda984 -> 4.6.16-libvirt-machine-controllers sha256:7531aba1073b6c7b06e4ab3b92e0ab05703ed74ba77a65bf566b2cd821978965 -> 4.6.16-ironic-static-ip-manager sha256:75ca081ba04bcf0ce322bfc541d2cc2ddd1f3b0c8825925358e7146f36ce7a7f -> 4.6.16-prom-label-proxy sha256:75f237fb87b4e370a719742f5de2d9f2d8ed0a048cd9d342f48f1ce14d88bffd -> 4.6.16-cli-artifacts sha256:77c4919a4a2e263cbad3fd35311762f2e3da0b2736ab6bb89ce4ded0a217b69c -> 4.6.16-ironic-inspector sha256:7952908ea6b485af01e78910f6f82e374cd5c983f7bb5b4d563e8f24be488894 -> 4.6.16-ironic sha256:7e22a7fb86bdd9ad0fe6f9e8a7fd1117ae498a9e87dc92a05ac0b7f7a2e912b2 -> 4.6.16-ovirt-csi-driver sha256:81860fff63f45155f131de179922a5f84b0ab82b20cc799d7e51fe5671cd2eef -> 4.6.16-jenkins-agent-nodejs sha256:87f0303178974ff58ae86a7c32aaa0a575b7c1460419649d50521f15be77c3a5 -> 4.6.16-console-operator sha256:8cbdddee540fbbb4ff2bbadfb2e8130e5e07cbdb10907c82f9f1dcf49eb79584 -> 4.6.16-cluster-kube-controller-manager-operator sha256:8f6a50fcca9a828869636b91acfd93d09f85b92c7391b7da290ca45672bbd26a -> 4.6.16-etcd sha256:912d4d25eaf6b370ba95cf563fda3fffb1d1d0c5f0a9985dd2d364386c33d0fe -> 4.6.16-tests sha256:920b27500df6322788e6647407457f8a9da5f8c6b45d5966c5d7acd0b2dcfb61 -> 4.6.16-kube-proxy sha256:97c99af2c53d8ba4dc369542763988deb1495d7dbe76358c691e3b22e93faf46 -> 4.6.16-openshift-state-metrics sha256:9a055d4818c7458f5332bc1740c5744c5d2eef6c3241ec65201301c3fc6b6615 -> 4.6.16-cluster-kube-storage-version-migrator-operator sha256:9b2cc7f532d3039d5a0619e27b93712617ad67c266198b16c8a1f6f9bb3d247d -> 4.6.16-must-gather sha256:9dd2275004cee4be18e3a71ef4aade9c6c35944d9d1ad7a043cf30843a419ea7 -> 4.6.16-multus-whereabouts-ipam-cni sha256:a283aa5a0208539b77ecb5b748ec2fc4f2de1d4549533cb800e5ee5369b71e73 -> 4.6.16-cluster-monitoring-operator sha256:a2e6e7f2eec9fa4ed67007d42a16532317b758f439ffd6ec6ff9aec370ff1233 -> 4.6.16-kube-storage-version-migrator sha256:a85d8ed0ff89f61484dc5aadb1a511649003c64da180fbff3948222468fcd51a -> 4.6.16-baremetal-machine-controllers sha256:a8717301e949cf6dbf145eb6941775ef76550bc26a8afbaf66ba109824f43b70 -> 4.6.16-cluster-node-tuning-operator sha256:aa54ec1e526be0a65c72245b6ff81224107b5a12f4ff2f4a8fd77a6ac840ca45 -> 4.6.16-csi-external-resizer sha256:aae15209d9d34b408c27f3bcab7b3810ad14513f991e242ee494989d367004d4 -> 4.6.16-tools sha256:adfa65c8b107fe43cdf6ae89525712870150fddbd573f575f8241d0b8cba739f -> 4.6.16-keepalived-ipfailover sha256:ae373b6a7cc81789514df9d0704bddd515acdb86e7fe3413e20d08939db343e7 -> 4.6.16-csi-livenessprobe sha256:b027890eec7b26fbc4d694a1d933b44e8998fea4c3821ec3af38918a8d759e1a -> 4.6.16-hyperkube sha256:b253d0cf853cbe222125a38779421462c3d014ca9229d00bb15976841060586c -> 4.6.16-csi-driver-manila sha256:b3066917a24ed2eae518582babfb557753745a25a3555eeaea44d63ea5e4df0a -> 4.6.16-gcp-machine-controllers sha256:b3aeabd7e99a8006dbbbbc86f6096aa16f60cf904689e16e03c2426184e1bb8d -> 4.6.16-csi-snapshot-controller sha256:b417eea4009c500effc1e11bcc66fc4ac1334708e9cbc0a67ba696d6948c6e2d -> 4.6.16-aws-ebs-csi-driver-operator sha256:b4722a2f94a885d133103b4f66f498cef04022dc941b2794968ffefab4aad260 -> 4.6.16-ovirt-machine-controllers sha256:b4c41a2fdc2c490851c44ca6d0cd340229d3d46cb95ea517b197355d9b679b81 -> 4.6.16-cluster-storage-operator sha256:b6a553024739ba532e2bc97003aa2f46b580a394f70dd54279b8be2d7d975612 -> 4.6.16-csi-driver-nfs sha256:c1a5188787e54134e10d7a197962f7188f75db167370a50877e0c86c854ff067 -> 4.6.16-cluster-image-registry-operator sha256:c354a72765cf7901bfb87be3838aa091d298afc342c997fe9497298cdb397098 -> 4.6.16-operator-registry sha256:c46a64f3033985acdae163507c499e5746fa933b4a9a28e741baf333b1adbfb9 -> 4.6.16-cluster-authentication-operator sha256:c5912419d58ea296f5c8a566c6013989c2618cc236251df8cd27ee8ecdbad4a2 -> 4.6.16-ovn-kubernetes sha256:c9543363a23dae0e8d6ae3d61e961f9af197ef14cda92826dc308dbaacca8302 -> 4.6.16-cluster-openshift-apiserver-operator sha256:ca475f58ef7e951dbd8cef593b13463b04dee52d7610bc27595ea5ca92652f9a -> 4.6.16-cluster-csi-snapshot-controller-operator sha256:ca7c22eb1f41414ab680d8aa9e0f4756f713cf7545277ad57b95248f4a770907 -> 4.6.16-ironic-ipa-downloader sha256:cc8af64e1c602d5a3ab8856722e4acedbcb4020d8b9d5567d99b16a74f524e91 -> 4.6.16-jenkins-agent-base sha256:cd948cbb88ad5b4659f7c029d1f0046cf3fef6d0a603328ce70dc72e17c78c13 -> 4.6.16-installer sha256:cf7ee380dae0dd1f3c5fb082e5b3809b0442dc9fe9e99bebb5f38b668abf54f1 -> 4.6.16-cluster-kube-scheduler-operator sha256:d1b83fc9be0c56a5228a665b2650b04788b183a613b24cfcdb791da64fc8a387 -> 4.6.16-aws-pod-identity-webhook sha256:d29b4fc60b9ae6b11aa1e3b2fc605b91178c2a6e70f576aba64989c8b597521d -> 4.6.16-azure-machine-controllers sha256:d801b556a675f3f128b320833c79991ec06d495ba56c978b9d616bf6ce9ba3f7 -> 4.6.16-service-ca-operator sha256:d8b42afa13c777273c5ad8a4be1dd38613ea8aec7b0af09c0593649347077f08 -> 4.6.16-csi-external-snapshotter sha256:db8252391e3b9ee2a956d36d610edc2c5b0523f7d42991799f2bac86b987f1db -> 4.6.16-multus-route-override-cni sha256:dbf686f41dffa6f2afb3569356c21119ab82494b4f62ed5caf02360e96ef4d82 -> 4.6.16-cluster-update-keys sha256:ded7909adfb34fbd75611af70b8715a6877d85dc865fd833e0a993df72eb81c9 -> 4.6.16-baremetal-installer sha256:e020e905363fcaa22207ae6b3cb5d24881eb0b7ab85977883b39d7296324331d -> 4.6.16-cloud-credential-operator sha256:e1be3c8767adc4b83416f54adcb6d3b35b17417e1ee0545e76e74d659d519087 -> 4.6.16-csi-external-attacher sha256:e22945d0527b98e4c2de2ef70c7c59b093137daff3fc505f45c8df165fca864f -> 4.6.16-sdn sha256:e24bfb27fee43bb5925b4157222f8ce62cc628985abf6895e2a58db10db0ac83 -> 4.6.16-cluster-machine-approver sha256:e4daa519e0360eba2c2335e687810b1effb247dc47a1ad66eb91b770d2f991e4 -> 4.6.16-cluster-dns-operator sha256:e777ec04395e501bb3e46f0551d40df14084f770555d2a65b5ee0ef09d89375d -> 4.6.16-cluster-openshift-controller-manager-operator sha256:e7ea15b0cc3d631a251d0a8df607de2ec2bfa13206933d0c52f009f55ea634a8 -> 4.6.16-cluster-kube-apiserver-operator sha256:ef64ecabdb35e74d88b0cbc76f90309eaf06ce417eaf12feb466acde7b4139c6 -> 4.6.16-insights-operator sha256:f3b16fb1ea070be814a631a70a664280ca18879b635fb012eb38584ab09f786e -> 4.6.16-pod sha256:f5c72a30229993db04204c04b98b0241d2f774cd4e162ea6fb9e6b84bf367d91 -> 4.6.16-aws-ebs-csi-driver sha256:f7d1dc09e3061dac704c65bf3d20fdac51239b7334ad4551c0731fc39e4073d8 -> 4.6.16-aws-machine-controllers sha256:f8763ef2638cdbd023bedc9c63c33ba77612a082f658d5ca86a01b61e6ca694a -> 4.6.16-baremetal-operator sha256:fc3a7a0c23a2aa1fb1130c0f8d3e94d626273a04ba1f3e672533090debfaf4a9 -> 4.6.16-kube-state-metrics sha256:ff7922378b3fc0ba3227b74d797cc9fc5bec67691a770490f83b13c642755676 -> 4.6.16-network-metrics-daemon stats: shared=0 unique=247 size=6.228GiB ratio=1.00

phase 0: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com ocp4/openshift4 blobs=247 mounts=0 manifests=121 shared=0

info: Planning completed in 23.14s uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:63abd44e70e49ffb7bab7df860d41e07f5ae408f0dcc68938d86f862d0783623 869.2MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:198df30fb540a39ff3d54610b33886dfbb76fb5d12941e06ff312d2d8b2baa4f 8.602MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:fdd39391986bf72d76f4a04aac2da35dc4c28ca4ca189b87b4382ac2b8a3f47d 28.08MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:493cac71a7b80928513997f8c69681618b1b0116ac5ee95c5e4eba0b932d0c86 86.79MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:9714581c5d74ab1090633c3c047dfc08beb750e568d0faaaa6d53c127f5ccc4d 60.85MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:fe0f36c58742b57ca7660fec7c338f2f5d1c4ac768dca0300aef4ed3c8f898b8 92MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:c91c00ff34b7bbaeec641bee65ba246b0cd1856756ac010e9acf7615c3bab154 112.8MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:fdfecce9efa3b64c89e4af5e774f5c6905c3ee58237c4bde63b74957eeafd283 17.12MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:bf7b90eec382c2a01da77860a6bfefd81409279590bc94f226ec063a379ee4aa 37.85MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:f34fc43a4dac72e83211c8af863211b717a196440ad377a154ff9868f16565ae 17.24MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:8027498bb7351c31f1da80cdfa3dfda7ae1302f61c642c4882a49850975fcffa 32.74MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:b8054f1b540e41b87156847c78212beac889e9a11d9e98441cc95a286b8b7d92 7.404MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:30b2af4f3f04deb220e8af1a727ffd52260bff4f3280ae12632a95cd9b3e8a59 14.71MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:36468b62fcd3c6c7bc0793eae7a13ac1d2a9757f0009673911be6610eafe3075 1.157MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:d845d5003cee057dd762e1626668675e10856f671f280ad9d3cc3d377ac470cb 36.52MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:ce3c51b51603a30638fffb0f4fc56961952bfb4b4d9a096daf928049cddffa34 6.535MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:d373fbe4ef281aed903d66cb6888422d49a555703daad17be45badaa95513fcc 44.12MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:5bed23e75deed901fecb501aee9337b35f4fd5d5862f2c1387bdb770a012410e 27.18MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:2b650abd7d8225d729a86f0fd81f5512514cdd6764fb8b0dce1b8902567322ee 9.027MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:a2510c607cba00019ab4485fbbe521350ac2fbb0cf919794d67a5e3bcb925a7b 36.77MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:0b12a65b7b9ee8ce89adb68c38120011dedefc0d0f5301f8d19310077bb883c7 27.05MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:b0ef12868506f33beb04ec505e179fc54664595be5cf8421ffb4d28753c4e290 21.59MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:54ad16f053f8c29152fca1353db0b23bda423dafb58704ac7e5d145e2c2b8552 12.43MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:78c5794215329e13b370e572b835f3856d1c80782997e5d794ed6f400b8cc9d0 14.26MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:19cddb0c9b275d3549292b78631e0c658cd6a75633cfff023de97f04ee4dfa3a 149.6MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:e129daff942e5eaaba6887c9ca1621e62b715158a684162be7f07589e0cf2961 477.5MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:f02dfcd805b96ef833ff8bd5464a5185f4a7b78e2905b304d399b4776e214797 94.94MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:987721fc9e66b1f9bbdf1d93950f03c5acbbe759bd5d5b35a540fa6f71298064 14.2MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:7e10e22d9cfac52cf531a5726c3b69559c3a730d8abe5696970eae06a5d6a075 103.1MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:3e4f6e27d6312a7306deb34d2b21fa9263a0686bd909f1984f939e80a3bc003c 5.918MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:d2cefff4288f0e4c8f119f80ffb3132b886c841504cc84b8a74a8cc20b58e700 75.43MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:2a34810165844b67388e08a8887b4c26e35eb6b5e59e9d78dea80f432a13c555 13.7MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:2cb776079e5183b0cf8d23efe7d9a4e007e6a915d8fa8bf399b7c627b39fd4b6 17.91MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:641085b680dfde549b8709396d8d71d1afa6003056de98628ad4f358400c5197 22.29MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:45f851bde7e1f1a9267bb6bb4a9a08d9117c8455a3ac1e94c8096b40e170e380 29.14MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:b06028a95508df2495b10b14f631f7376bec91e7e3928c8d4730cff7c43c75ab 25.28MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:3491a0460b37df32f4ce28919349f59efdaa07320c8f550f71d1de1ab84be7c6 20.87MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:5297e07497ce1d2a1a79d7359fcd03a5ca47f5f7ba4f6c8a5479518f51f7875b 429.8MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:ca570ef983da226c8117457cfd8df66483cf39e9b0be2a76ca32499e0f840f31 7.995MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:6f406e2413170ad6cc4597a74c0c47d876370aaa65c24070f8fbcc35cb4f7ed3 74.27MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:4061672a2b58d2c95a79b774938d9ed40e490491d6a3c995c746e43ee44b373b 5.522MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:52ec59fa0941954218b1f8877c44785790e3f3991afd738a359e2fa68d23a4e3 23.2MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:2755828e3e9f023713dbba02734f4fbd06bfa9edaf979514eebc9d07fc5dc5c3 81.65MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:e23e2a16c5547563c88b3247461c0a05a59badad77d29ed47b54ed16840c48d6 23.23MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:b16b5043391c7b6db273a2eb06ebc2e99257089e52f30c7fa2711033d97ad668 19.08MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:a498b216fe63860e105f99ded9fa2fb99dc471f03f07c477cff0f178ea9f123c 19.92MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:022d193218d77ac322a1a77501a86982be712612dc9286250ef60d12e368ed2b 36.93MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:c83ff2e8ea1136235688c0f5e0e9d9d8e5703439783707f9d3e3643688b04f59 15.43MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:18e9172ad4d5dbe29de81e57f12999f337091dc3cd5cd9977d1be7d2b2af0b32 15.28MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:5549446e3c52e9c866db0f259f491f22e9a85e823e47afe2eb2f31b9e0dece4b 5.086MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:ef8d5a3f24626311ccaed51b09ec0ace59a8d3220d715e4393d10ae7eef5cf4c 217.1MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:23cd0dd42cfa30801cd1b09b7ab6903a056612b14ae9111a8acf000a70425968 127.7MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:44a89d0c0e0a0953ca915dd70bd714f612fd5a6e4e1116d306eec61a1393eabf 18.62MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:aa7fc101f2b03cd851e7150db8de40d4ea3d7ce46f74dd601c3dcac04961e8de 12.76MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:dc39e49ccb938880884668b1636945a4f73697a603995993d1458d388f2097a1 43.06MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:95efbc6c0025e6746c9c573fafb81c4af15895e756b1d06a8d0092ebdd7bf10f 74.96MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:caf27dc5812fd1b7fb8242ff15095f1c6b8fc102d115f0af95bd5fd375aa8b08 41.64MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:4822868ec24440f79e687eb033eb354ea7c362b00ffffc667d46f1fed2f7c59a 35.38MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:d1130823979ed7f8cc1a1dbb1af51d58777556bac9c0b41f487f5a60783e60c1 13.94MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:7d010bb4749572e44924b52cc66faee554b720f3d500d0f44a1963e4e7ad2664 190.1MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:d59ef1c4d9e1d38ed9bc3c1cf1f66724f099e125f841ca6bfcdf78d65fc88f08 211.7MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:ab7046598ec421a9f6fc383dd1e22aecee86c7e8beeab85662035072d5ee819d 10.86MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:310c83d07e46ee797d4e086c894d2215f00df7915a2ad2a4bd28960dfbfebbbc 39.18MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:b722bf8bea9f0249f2fe93647db16701ca78bf9d33c1e98c599741d42977585a 20.74MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:2a882f4331bbc7ab364c4c8788a8181a26d9499b0a8cd3ce1c512b37dd52a06e 10.58MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:13ecc66841369ef97526de74a483933d92feafbc5610a8de43f1425de2bfd6b2 15.36MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:e5ceb5aa87feeac9c0c67363c31d2fd2033d789db1014b9d86796c88dc47bda9 97.16MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:07911f4a50fa95f17dece4e4b6e5ca137edfd5cc6b8a734884b2744f1b9f951b 145MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:6ba8ea8de6a2464c29827e4bbb7a08223c9295bfc7b9fc2d5969a8d5a4ba50e0 17.94MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:90fdbc5e6d8e3313a268129c4f170e8e432834601b36f51ee0709b84dab634d4 8.134MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:5a8f76dad18a9fdc16ac6a4f430ce92a26ba4ab94800c1d27d67901f3e044052 17.66MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:cb58464b68e86cd1b59a21c093c8769fce34ae42554de001eb5ff99350696e87 18.83MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:2fd83570867e7782acf1933de4d0ef9e8e5211c22d7851cc0c025f56b30ecb75 32.48MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:bb3369a5a0139a29dfca06cb3d881796ffa2aef2eeee8266383d93372da8e45b 119.3MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:4151cbe3a62535e862a9f25114a0ea2edb4b8537d898ba8d52f67d5bfe62812f 145.2MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:fbf81b9a2d44e52bc19fe30813f3c31a701d405ccfd53724cd89afef1c61d2f6 8.335MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:301430eddadca205bb3935e39b9bea081ede0ec94f60eab100055b03d28f885d 14.34MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:bf82e58856a3f282ca30f5d3b13a7597292d18dc8193940005c4fc483481d3fb 16.1MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:d987afbc5509d81499d81e77c483c95bb38e10ff2e5797a04df4f839310c8bea 14.08MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:bfc5b1853a0d337329002518f45b36300aee5ae8c7ca1a0b3dbb2f807cb62b48 27MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:9707c00490a94cd0480a598759e1b7b3972d623808aba696c71f99074c2837b6 15.26MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:03242baf3a888b0d9c0d22f3220be8b83e2e3476113c1c77f90fa0271fc8843e 14.85MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:71ab386e206d48a44c314df009629d597484ec4f367bb4d5aef20f194a01cb7c 17.4MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:db0d4a0292552c80250c4e8141b14bd059baa54c43c0cdec37042dae69714ba4 13.58MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:ec51190d499c00c0031569c0e2d4b5de3985317a6eb07c34b3cc09059010c0e9 16.02MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:71cb781fe003a4244c559239d5569fad64c82e0fb96c6e0bdf54bbd41cf6c27f 44.56MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:f4f4cd736d3eed339a3a768f6dacb9be3b5a4f04feee89b45281e76530fb426d 28.41MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:28786b00e405505def8342ec59d4063cad2a477b566c33106c4d81f81d932aef 3.427MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:658ea0558e8fc93b1d761e980b0daa6c97a213d9c648e058ec37c47f987783b1 5.127MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:6c1171a658062d861d044d5a65677b3036803af79895ea795f01dba448c32e06 30.97MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:cf76770be7dd07268e8bb8bd85dc315bba34d496b2587587a81287527154c9df 52.61MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:db3638342a1cc20d72b3e0a4ae3e0bbdf57e4d553281514197fe2d3639582099 26.59MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:747dc137594d496704ece95c709d3e06c2afc5a67677f061c7fdc9028c0764d8 75.77MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:ecb87e445723ff7a1f695b2c93100b907ad0be34649f25c326c5c94533288b8e 22.87MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:e78ab868aa58d652199ceee490cabf1a75946309dac64e6e7fe7e64fe885f93d 32.7MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:e47b394b2dc87ee5b91eaa257aa8e6c3aa80e238da5aa1920333d64068d1d88c 117.7MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:c90142f6e9b705dbf579fe16d65d5be6f5baedc8742de25b40ed8916d805590a 19.35MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:491c4e0b3ff186a5add54eb3d215ed43caec32b3ad750a51d2ce70dea7611d29 28.93MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:19cd21687bde9883b3efe4b17bfe296829fb44a5b9dce0c3119ffcd31394f58f 19.49MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:3b66af967542f8d2e5c293554421e9c16abe78f86148a92f730bf55131ec83f2 23.45MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:5a38303e3fff29c38288480ce733cc71d2005b0844a1dbf53fe243403d4c71ef 81.7MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:c515503b8e2819a6e729ba73ca3caf6a36a41d56dfe237de1b6e4229a3706a1a 12.93MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:a972e3f4c83ab0ab8de9473c498b0d04125ce22874d8812128b3d8c83eff968c 121.2MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:bb68e8edd5efd73e622c6c623356c6aebf296121387195f74d08a65965f02bbe 9.153MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:5142b5c90cb446401eaafbc7a5d8c2270a201c78020667665490b0e37a5e292b 24.23MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:f937b750e970a9573caa4f6bcfa21a897581cd7e210ee2032dc76b238bb6acce 14.29MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:d489fd3245f8a64076e4272d393ae36e83dfd6fa882abda580c41dc456c74444 14.45MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:89b397c96bf4c337f1a6aa1f06923e03f5c5694344a75a4ec9c41edf83b1f320 35.51MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:f8411f1ef5b9aba2789932ecb6a86a981cd5411aae951940733bf8b8c405fc8d 14.26MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:e15714bf0b39f1ae49eff9902d3065554ed1e9494860f2df2a116aab0091ff22 17.88MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:fa2e341b8efcf6a3d3c119baf2c02cbc19aec128929278b64cc9ae1addc8b44b 13.7MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:64c84472ff54efa0723bd2c8dd337d9bc5637cc2be9c58457dea6119022ba23f 14.53MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:a4428ac7e7770b7d06099cb24139ee8870dd811e8edd29b81d6ecf562ef3b18a 10.62MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:3fa171eabb0e3e053d34e2ffc5a8fca873296f9c031d2fae187bf87c22401d8f 15.35MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:41cf0e9a4669634d918026d5e7f9a198ec04084ff17d9e6003ca888baf9807f1 27.95MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:06f855db5119587360772750b49d596c59e74160fcb6c609e69d846ae604f348 2.837MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:bd7a6a63db8b0577c6804b9c899de40cfa58f3405628c7a442de9ed073abca2c 15.06MiB uploading: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 sha256:f93c76b49a2eea09f023f71d2c27b626fd98b7b6f112afba8920a5e2138fd85f 21.71MiB sha256:cd948cbb88ad5b4659f7c029d1f0046cf3fef6d0a603328ce70dc72e17c78c13 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-installer sha256:01548ee4f07a6cd9167849a2c2c95e0a64793579856ea053c06dcf1183b6c74f vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ironic-hardware-inventory-recorder sha256:505351039d9b93edade905f688e25d827e5d36b3e81878b927a47b7fcd7a1df2 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-openstack-machine-controllers sha256:63b7992c8e200d7669bc5b81bcc1f4778fde9c65aca6ae026ed1a32bcfaa18e1 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-telemeter sha256:aa54ec1e526be0a65c72245b6ff81224107b5a12f4ff2f4a8fd77a6ac840ca45 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-external-resizer sha256:b4c41a2fdc2c490851c44ca6d0cd340229d3d46cb95ea517b197355d9b679b81 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-storage-operator sha256:624f99bcdc9fa53c0b48c7be6fb54056570201cbb5b5bad8b4029055235e037f vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-local-storage-static-provisioner sha256:81860fff63f45155f131de179922a5f84b0ab82b20cc799d7e51fe5671cd2eef vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-jenkins-agent-nodejs sha256:11c7c2604386ab6c5d7a5e5e7579abc20a51c57755faa1d995528f5096a1af27 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-deployer sha256:690849c8f2824839e8730f0aa909e26f5ffff9f92355ef9e63b51441b9d8a2b4 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-external-provisioner sha256:c9543363a23dae0e8d6ae3d61e961f9af197ef14cda92826dc308dbaacca8302 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-openshift-apiserver-operator sha256:cc8af64e1c602d5a3ab8856722e4acedbcb4020d8b9d5567d99b16a74f524e91 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-jenkins-agent-base sha256:6f5ae49f0d437e0f9ad25d5342576f8a7b052f9d676d926c35c92f2d79034cc4 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-kuryr-cni sha256:8f6a50fcca9a828869636b91acfd93d09f85b92c7391b7da290ca45672bbd26a vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-etcd sha256:6edb934ac430fcd1a2da4f5ba4ed2a2f5e5017e7ba970c1fcf44b827f0a05776 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-thanos sha256:f8763ef2638cdbd023bedc9c63c33ba77612a082f658d5ca86a01b61e6ca694a vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-baremetal-operator sha256:2a7d2cd3ae6e1d2e0bab3682f86f916ca93d93aaedb04cff478a72063557f090 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-operator-lifecycle-manager sha256:5dc1aaa71109b1cfb8e77402ae1db78ac1b08ec1b136e55ea8c4ba90cedca320 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-samples-operator sha256:9a055d4818c7458f5332bc1740c5744c5d2eef6c3241ec65201301c3fc6b6615 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-kube-storage-version-migrator-operator sha256:c5912419d58ea296f5c8a566c6013989c2618cc236251df8cd27ee8ecdbad4a2 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ovn-kubernetes sha256:15a7862dc4854599ee29c0541f49b47d21d24ca1a3ce0fd11d629d8e48ae035c vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-policy-controller sha256:aae15209d9d34b408c27f3bcab7b3810ad14513f991e242ee494989d367004d4 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-tools sha256:c354a72765cf7901bfb87be3838aa091d298afc342c997fe9497298cdb397098 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-operator-registry sha256:171385f90bc48fc195d0409dc8f69e770baf51bb1cce6747df8f10596d0462cb vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-operator-marketplace sha256:1d21de46a169dda2c8535058f214fb7d910f65d765543658fc56c55103d8b250 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-installer-artifacts sha256:09e3228d294d07940efebc06301179cf376f50ca16efe21ba0e343ab12ffa131 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-machine-os-content sha256:912d4d25eaf6b370ba95cf563fda3fffb1d1d0c5f0a9985dd2d364386c33d0fe vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-tests sha256:44cd3de006e68c454292db866b16417bd7dd77d0becfd2b30593b698520b08b5 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-bootstrap sha256:ff7922378b3fc0ba3227b74d797cc9fc5bec67691a770490f83b13c642755676 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-network-metrics-daemon sha256:6f6b06d31391d43e69c30d56a95677b32730da48160562e49911d1c1b14b034b vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-container-networking-plugins sha256:75ca081ba04bcf0ce322bfc541d2cc2ddd1f3b0c8825925358e7146f36ce7a7f vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-prom-label-proxy sha256:4d6d9e55b48edaeba59091abbcee46162b9153d44b9aac30c98ac94c6bd085cf vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-coredns sha256:731c0bf07c47678465a2f9b344f7f94de4f6990538411ced28f8f07fc0bda984 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-libvirt-machine-controllers sha256:002441e0c2d60adca78f0b063dfc7594503735c0c913105588eb5f1393fd1b6a vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-console sha256:48c2c43befaab410dcd1b296228150b8a1777c6ad0bdcd194e0a9087be3a2a6e vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-jenkins sha256:1ca8bc8e75c39d96959565dac7690172f6caf3468c83123589874f6dd9ebae5a vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-prometheus-config-reloader sha256:65990aba7cc00e680b34fcd83e9cc9704d45cd534f8c85232934f560f24281a0 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-prometheus-alertmanager sha256:d29b4fc60b9ae6b11aa1e3b2fc605b91178c2a6e70f576aba64989c8b597521d vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-azure-machine-controllers sha256:fc3a7a0c23a2aa1fb1130c0f8d3e94d626273a04ba1f3e672533090debfaf4a9 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-kube-state-metrics sha256:70b792bd18095d067b20dbf79309e53379f9a039ef4dc8b8e065bb209a8f0a33 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-openshift-controller-manager sha256:3e855ad88f46ad1b7f56c312f078ca6adaba623c5d4b360143f9f82d2f349741 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-x86_64 sha256:9b2cc7f532d3039d5a0619e27b93712617ad67c266198b16c8a1f6f9bb3d247d vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-must-gather sha256:60a313ac29046e0e0c31679a4b2187f94ee389e647b6b98bd72bc62b32632ac9 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-etcd-operator sha256:a2e6e7f2eec9fa4ed67007d42a16532317b758f439ffd6ec6ff9aec370ff1233 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-kube-storage-version-migrator sha256:4460e0ce7e047b86b9ad63d56cf0b953b419b47bd851b07e7ce9aa3deac64d86 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-mdns-publisher sha256:c1a5188787e54134e10d7a197962f7188f75db167370a50877e0c86c854ff067 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-image-registry-operator sha256:f7d1dc09e3061dac704c65bf3d20fdac51239b7334ad4551c0731fc39e4073d8 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-aws-machine-controllers sha256:e7ea15b0cc3d631a251d0a8df607de2ec2bfa13206933d0c52f009f55ea634a8 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-kube-apiserver-operator sha256:71e1ed11e92fa647249ed4efbd53679ca459466761efc926baf7d2e4583e4562 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-configmap-reloader sha256:23d436573130137206e222a072e20d4769bf92f32a6a93772e1074a290c2ddad vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-ingress-operator sha256:ae373b6a7cc81789514df9d0704bddd515acdb86e7fe3413e20d08939db343e7 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-livenessprobe sha256:ca7c22eb1f41414ab680d8aa9e0f4756f713cf7545277ad57b95248f4a770907 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ironic-ipa-downloader sha256:7531aba1073b6c7b06e4ab3b92e0ab05703ed74ba77a65bf566b2cd821978965 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ironic-static-ip-manager sha256:18c9d24bdd566275c120a978835af9a14d66052352391113650982e47ac70387 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-driver-manila-operator sha256:7952908ea6b485af01e78910f6f82e374cd5c983f7bb5b4d563e8f24be488894 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ironic sha256:e22945d0527b98e4c2de2ef70c7c59b093137daff3fc505f45c8df165fca864f vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-sdn sha256:e020e905363fcaa22207ae6b3cb5d24881eb0b7ab85977883b39d7296324331d vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cloud-credential-operator sha256:87f0303178974ff58ae86a7c32aaa0a575b7c1460419649d50521f15be77c3a5 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-console-operator sha256:d8b42afa13c777273c5ad8a4be1dd38613ea8aec7b0af09c0593649347077f08 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-external-snapshotter sha256:7e22a7fb86bdd9ad0fe6f9e8a7fd1117ae498a9e87dc92a05ac0b7f7a2e912b2 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ovirt-csi-driver sha256:599957b272e4012557e5792ad5a521fc53b4c7ec277ed888e8cbda40e5f488fd vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-docker-builder sha256:920b27500df6322788e6647407457f8a9da5f8c6b45d5966c5d7acd0b2dcfb61 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-kube-proxy sha256:9dd2275004cee4be18e3a71ef4aade9c6c35944d9d1ad7a043cf30843a419ea7 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-multus-whereabouts-ipam-cni sha256:b4722a2f94a885d133103b4f66f498cef04022dc941b2794968ffefab4aad260 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ovirt-machine-controllers sha256:d801b556a675f3f128b320833c79991ec06d495ba56c978b9d616bf6ce9ba3f7 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-service-ca-operator sha256:ca475f58ef7e951dbd8cef593b13463b04dee52d7610bc27595ea5ca92652f9a vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-csi-snapshot-controller-operator sha256:b027890eec7b26fbc4d694a1d933b44e8998fea4c3821ec3af38918a8d759e1a vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-hyperkube sha256:3496c77d7281fd4854732c9b8a0eb09283a162ce0a4b8a3c96ba51184049f0be vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-haproxy-router sha256:44e20aa95a63f06449264a1ac0165c5311dab18840c82e6e4533d6bd38670772 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-prometheus-node-exporter sha256:a283aa5a0208539b77ecb5b748ec2fc4f2de1d4549533cb800e5ee5369b71e73 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-monitoring-operator sha256:b3066917a24ed2eae518582babfb557753745a25a3555eeaea44d63ea5e4df0a vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-gcp-machine-controllers sha256:f5c72a30229993db04204c04b98b0241d2f774cd4e162ea6fb9e6b84bf367d91 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-aws-ebs-csi-driver sha256:adfa65c8b107fe43cdf6ae89525712870150fddbd573f575f8241d0b8cba739f vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-keepalived-ipfailover sha256:c46a64f3033985acdae163507c499e5746fa933b4a9a28e741baf333b1adbfb9 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-authentication-operator sha256:3b8dc85390742e2a949639a410188a9202f9fffce8a32d8131fe428e79b89a70 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-kuryr-controller sha256:3899b8fba1ab11d5cbff5581ca3bc7c06ba47eece58588cd53f93ead60e8ea4c vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-baremetal-runtimecfg sha256:1dde6c31e71824d17329beaac7c87ff5c5b1d7c1f0a272ec858fec1812e775c6 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-version-operator sha256:21d08b20bc5fba9f93ac98a2e81d933e3b0207d002bba0966e534625efcb3e09 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cli sha256:2401d0e248e71a21a3db7a9e217130de949ee9c7ebc987af518537dfb68dcaec vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-machine-api-operator sha256:cf7ee380dae0dd1f3c5fb082e5b3809b0442dc9fe9e99bebb5f38b668abf54f1 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-kube-scheduler-operator sha256:2b5ee448395420861a3e80bce64780e809fd500b061bd40175f33708055aa985 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-multus-admission-controller sha256:6eb536df1ab94d857c363da104c5d47a37c11a0b8edd2a0e2b04f8437996ae75 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-jenkins-agent-maven sha256:3a776866d39bcd264d389b75616c9dc32e6af7ec740192d252600fc1da63a488 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-prometheus-operator sha256:e1be3c8767adc4b83416f54adcb6d3b35b17417e1ee0545e76e74d659d519087 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-external-attacher sha256:2c71b04a8b2bc5c9b97ff160cf8a1814c41945211e1484ce7a4a83c5cbc06769 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-oauth-proxy sha256:8cbdddee540fbbb4ff2bbadfb2e8130e5e07cbdb10907c82f9f1dcf49eb79584 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-kube-controller-manager-operator sha256:45f89b0f1fc62eded3ed3482f5574d41f374ea2e463b0c587f32dccb6f2f127f vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-network-operator sha256:4491eb6118d48f2c87c313464f6cccb8c290fb855df4446f2d3c37119f5b102e vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-autoscaler sha256:b3aeabd7e99a8006dbbbbc86f6096aa16f60cf904689e16e03c2426184e1bb8d vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-snapshot-controller sha256:71728988bb976b12fc435ef06b02e4822a89632f6136d96943f81009869d114b vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-kube-rbac-proxy sha256:e4daa519e0360eba2c2335e687810b1effb247dc47a1ad66eb91b770d2f991e4 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-dns-operator sha256:e777ec04395e501bb3e46f0551d40df14084f770555d2a65b5ee0ef09d89375d vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-openshift-controller-manager-operator sha256:5aad4db5d16c0c85ee1972184e0cb9e7ade4ae9f8a34044a9a306c684aa84e00 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-docker-registry sha256:1fc73f923f35676b887dcd25500a5ab24aa306f87ad885a546ae0235203c14bb vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-grafana sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-machine-config-operator sha256:f3b16fb1ea070be814a631a70a664280ca18879b635fb012eb38584ab09f786e vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-pod sha256:5696970cb8306c3028d6160a8ff02b03588c3323154473f4fd374900c918909d vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-oauth-apiserver sha256:2e6b62f9983aa08086e959694e3567bc269e42608d7c0de58ba571b10973e500 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-k8s-prometheus-adapter sha256:e24bfb27fee43bb5925b4157222f8ce62cc628985abf6895e2a58db10db0ac83 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-machine-approver sha256:a85d8ed0ff89f61484dc5aadb1a511649003c64da180fbff3948222468fcd51a vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-baremetal-machine-controllers sha256:3fcddac914fca3cdf0884aff6a21dc45ac83e1a5b94fc419b5ef54ff3647e12f vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-openshift-apiserver sha256:b417eea4009c500effc1e11bcc66fc4ac1334708e9cbc0a67ba696d6948c6e2d vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-aws-ebs-csi-driver-operator sha256:77c4919a4a2e263cbad3fd35311762f2e3da0b2736ab6bb89ce4ded0a217b69c vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ironic-inspector sha256:ef64ecabdb35e74d88b0cbc76f90309eaf06ce417eaf12feb466acde7b4139c6 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-insights-operator sha256:65d1479d2c780938afc97aa642c0d40299893c9616c2763247041bc30ec0c364 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-config-operator sha256:13640217da2ae26debbd19c124adaee503a5ab077d3950450fccd76dfe0941b5 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-autoscaler-operator sha256:97c99af2c53d8ba4dc369542763988deb1495d7dbe76358c691e3b22e93faf46 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-openshift-state-metrics sha256:59ea6df65ba624a0421e050777be4a229d2b66c5ee741adb38d4b1e2eb939035 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-prometheus sha256:4cff4fb61970d98c666f3e1a6fb4d0c6a25e557c1344b281e08e605235250ded vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-oauth-server sha256:1b2a3b09b8d6851c10cbac36362043a2446acebbe9f5d45b3c800cd49c3679d3 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ironic-machine-os-downloader sha256:ded7909adfb34fbd75611af70b8715a6877d85dc865fd833e0a993df72eb81c9 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-baremetal-installer sha256:b253d0cf853cbe222125a38779421462c3d014ca9229d00bb15976841060586c vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-driver-manila sha256:21d669ab48ea20f11c540cc030fde3f09426c1edbcbf573c8b44e0c72bcd873a vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-ovirt-csi-driver-operator sha256:d1b83fc9be0c56a5228a665b2650b04788b183a613b24cfcdb791da64fc8a387 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-aws-pod-identity-webhook sha256:75f237fb87b4e370a719742f5de2d9f2d8ed0a048cd9d342f48f1ce14d88bffd vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cli-artifacts sha256:a8717301e949cf6dbf145eb6941775ef76550bc26a8afbaf66ba109824f43b70 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-node-tuning-operator sha256:395bb85d2a03330089eedc10d6c0f83137eb7996cc842a94ae16ba49b365eb6d vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-multus-cni sha256:db8252391e3b9ee2a956d36d610edc2c5b0523f7d42991799f2bac86b987f1db vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-multus-route-override-cni sha256:dbf686f41dffa6f2afb3569356c21119ab82494b4f62ed5caf02360e96ef4d82 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-cluster-update-keys sha256:43a56fd67737795384bdd7b8da9763780c6aec6b0fcf39d78c25378e35ff0ce8 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-node-driver-registrar sha256:b6a553024739ba532e2bc97003aa2f46b580a394f70dd54279b8be2d7d975612 vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-csi-driver-nfs info: Mirroring completed in 3m30.18s (31.29MB/s)

Success Update image: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4:4.6.16-x86_64 Mirror prefix: vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4

To use the new mirrored repository to install, add the following section to the install-config.yaml:

imageContentSources:

mirrors:
- vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 source: quay.io/openshift-release-dev/ocp-release
mirrors:
- vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 source: quay.io/openshift-release-dev/ocp-v4.0-art-dev

To use the new mirrored repository for upgrades, use the following to create an ImageContentSourcePolicy:

apiVersion: operator.openshift.io/v1alpha1 kind: ImageContentSourcePolicy metadata: name: example spec: repositoryDigestMirrors:

mirrors:
- vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 source: quay.io/openshift-release-dev/ocp-release
mirrors:
- vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4 source: quay.io/openshift-release-dev/ocp-v4.0-art-dev

pmallan commented 3 years ago

@yrobla I could mirror 121 images as you see above. Please find attached inventory file and installation log file. I have defined these in the inventory file Inventory File:

Parameters for a Restricted Network installation

use_mirror= True mirror_certificate = "{{ lookup('file', './mirror_certificate.txt') }}" mirror_registry=vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com

Installation failed due this error. Any idea why it is not making use of the certificate provide in the inventory. inventory.txt test-mirror-cluster_76cd991d-a014-4fa2-b9a6-3e7b73625a56.zip

May 21 08:56:59 mgmtacm-worker-2 installer[4821]: time="2021-05-21T08:56:59Z" level=error msg="Failed to extract ignition to disk" error="failed executing nsenter [-t 1 -m -i -- podman run --net host --pid=host --volume /:/rootfs:rw --volume /usr/bin/rpm-ostree:/usr/bin/rpm-ostree --privileged --entrypoint /usr/bin/machine-config-daemon quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 start --node-name localhost --root-mount /rootfs --once-from /opt/install-dir/bootstrap.ign --skip-reboot], Error exit status 125, LastOutput \"... ing manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized\"" May 21 08:56:59 mgmtacm-worker-2 installer[4821]: time="2021-05-21T08:56:59Z" level=info msg="Trying to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2...\n" May 21 08:57:00 mgmtacm-worker-2 installer[4821]: time="2021-05-21T08:57:00Z" level=info msg=" unauthorized: access to the requested resource is not authorized\n" May 21 08:57:00 mgmtacm-worker-2 installer[4821]: time="2021-05-21T08:57:00Z" level=info msg="Error: unable to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: unable to pull image: Error initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: (Mirrors also failed: [vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: error pinging docker registry vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com: Get \"https://vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/v2/\": x509: certificate signed by unknown authority]): quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized\n" May 21 08:57:00 mgmtacm-worker-2 installer[4821]: time="2021-05-21T08:57:00Z" level=info msg="failed executing nsenter [-t 1 -m -i -- podman run --net host --pid=host --volume /:/rootfs:rw --volume /usr/bin/rpm-ostree:/usr/bin/rpm-ostree --privileged --entrypoint /usr/bin/machine-config-daemon quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 start --node-name localhost --root-mount /rootfs --once-from /opt/install-dir/bootstrap.ign --skip-reboot], env vars [PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=xterm HOSTNAME=mgmtacm-worker-2 container=podman HOME=/root], error exit status 125, waitStatus 125, Output \"Error: unable to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: unable to pull image: Error initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: (Mirrors also failed: [vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: error pinging docker registry vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com: Get \"https://vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/v2/\": x509: certificate signed by unknown authority]): quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized\""

pmallan commented 3 years ago

    "image_info": {
        "created_at": "2021-05-21T11:48:32.432Z",
        "download_url": "http://172.29.231.89:8080/api/assisted-install/v1/clusters/fe5bb79d-24b6-4a42-9ca6-f46d5fd601de/downloads/image",
        "expires_at": "2021-05-21T15:48:32.432Z",
        "size_bytes": 918552576,
        "ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+dWhOp1/82HUcIn1nNs+7+bmXICzMA/1FL7un/MTImGpQInhMPoi/bhGIpIBRiGB6spcGafM+1sVTSj//oOIH/EN+2C0RzbEGhj1YU8pawf3B8eTLXQJbS+IpY8K4JVbXyyTamQbf9pXddvjZqtZWy4QA5Z2hWYENdcQyJiSjVIP6BTsUKqDJdp4vm3QT53bX3BUWgkTFXImCRnVNCltqvUk0HuRJeL8mA+MrTp0GELHdJDQo0210QOFvmAjcf9DOxoYoGthi0KcAtWskS+Lwxf6IC6ajDsKQoJV59dmn0xDJMcS+W8Ha0+4PE9baIrX6/0QFgaCgDbbieRN579vH",
        "type": "full-iso"
    },
    "ingress_vip": "172.29.231.239",
    "install_completed_at": "2000-01-01T00:00:00.000Z",
    "install_config_overrides": "{\"imageContentSources\": [{\"mirrors\": [\"vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4\"], \"source\": \"quay.io/openshift-release-dev/ocp-release\"}, {\"mirrors\": [\"vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4\"], \"source\": \"quay.io/openshift-release-dev/ocp-v4.0-art-dev\"}, {\"mirrors\": [\"vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocpmetal\"], \"source\": \"quay.io/ocpmetal\"}], \"additionalTrustBundle\": \"-----BEGIN CERTIFICATE-----\\nMIIEVjCCAz6gAwIBAgIBAjANBgkqhkiG9w0BAQsFADBpMWcwZQYDVQQDDF52cmFu\\nLXF1YXktcmVnaXN0cnktcXVheS1vcGVuc2hpZnQtb3BlcmF0b3JzLmFwcHMubWdt\\ndGNsdXN0ZXIucnRwLnJhbGVpZ2guaWJtLmNvbS1jYUAxNjIwNzMwNTY2MB4XDTIx\\nMDUxMTA5NTYwNloXDTIyMDUxMTA5NTYwNlowZjFkMGIGA1UEAwxbdnJhbi1xdWF5\\nLXJlZ2lzdHJ5LXF1YXktb3BlbnNoaWZ0LW9wZXJhdG9ycy5hcHBzLm1nbXRjbHVz\\ndGVyLnJ0cC5yYWxlaWdoLmlibS5jb21AMTYyMDczMDU2NjCCASIwDQYJKoZIhvcN\\nAQEBBQADggEPADCCAQoCggEBALuV9PruSfpbL2nkcSFf9FEI5tmaGASLxop1u58l\\nI1vYXN23r895aZTF5OulGSWFCIDjWxMx6L/DNiX0mXtuVzwtBNKnTS/upqSmREZA\\nQb+d0vFSZ74qYFfOsnbnBSQV3EnQBKK0i7TtppQMK7LlTAZTE657yJyk63D0/SK0\\n7N+zzx5iA3RbAzc0XibkT7PoV3HM5Ul9vVdy2SNQl6UJO47tHn05+waRq6y33hzk\\nUWcPkfeSGhrYIWB2fC5q9OIw7X6uD1YDdOhqsQFOGNblH+N3px+TYzKfy/FSpJnm\\nCBLG/f+k6E9QAj94CS9rn6LDxLfBCFH17xkkpv4cLsBSS18CAwEAAaOCAQowggEG\\nMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8E\\nAjAAMB8GA1UdIwQYMBaAFGQVxMKThr9vfUUHTQkhSoJlXiiyMIGvBgNVHREEgacw\\ngaSCUHZyYW4tcXVheS1yZWdpc3RyeS1xdWF5LW9wZW5zaGlmdC1vcGVyYXRvcnMu\\nYXBwcy5tZ210Y2x1c3Rlci5ydHAucmFsZWlnaC5pYm0uY29tglB2cmFuLXF1YXkt\\ncmVnaXN0cnktcXVheS1vcGVuc2hpZnQtb3BlcmF0b3JzLmFwcHMubWdtdGNsdXN0\\nZXIucnRwLnJhbGVpZ2guaWJtLmNvbTANBgkqhkiG9w0BAQsFAAOCAQEAXKg4OVRo\\nzj5+J64/pxnKULh28caoiGZZjebtSLBUKBQCLc2gq8EzFwb6vnokwvGqSUU+KjVn\\nn+Mn2cTD5jX/G/q8K+HmaWRHu+SEwTc/PZ5JIzATOdc6RhaojiaGKzFYKESY0sn4\\ndBDOugU5nQ6yZPoREeHLgQQ6x1YJ9i7iNE0F8e6T7jQqbnK2YWPSJXWi8fFqJ51F\\n3i6ASEFAwqz8GGdpwBlxRtMNtE2XMqLZZwl5hKyYL1G/64E9oDeH1rENxsgYu4VJ\\nqrSxrXzxEcLTS4tEWQh+TQA2uA980csvSNyOagI4ccYlaquRqqBoDDHI3uvScT5+\\nat6nJCwtFav1nw==\\n-----END CERTIFICATE-----\\n-----BEGIN CERTIFICATE-----\\nMIIDjzCCAnegAwIBAgIBATANBgkqhkiG9w0BAQsFADBpMWcwZQYDVQQDDF52cmFu\\nLXF1YXktcmVnaXN0cnktcXVheS1vcGVuc2hpZnQtb3BlcmF0b3JzLmFwcHMubWdt\\ndGNsdXN0ZXIucnRwLnJhbGVpZ2guaWJtLmNvbS1jYUAxNjIwNzMwNTY2MB4XDTIx\\nMDUxMTA5NTYwNloXDTIyMDUxMTA5NTYwNlowaTFnMGUGA1UEAwxednJhbi1xdWF5\\nLXJlZ2lzdHJ5LXF1YXktb3BlbnNoaWZ0LW9wZXJhdG9ycy5hcHBzLm1nbXRjbHVz\\ndGVyLnJ0cC5yYWxlaWdoLmlibS5jb20tY2FAMTYyMDczMDU2NjCCASIwDQYJKoZI\\nhvcNAQEBBQADggEPADCCAQoCggEBAMUCZs5rN9N9/crlrp8ylVfKbgU+cMuMVgck\\nn5YmOPY/4WyiQan8IS6iL+fNn+3IV8Gvraoex2T2gL9UUV3bzQXLzTgqEGGjUEuk\\nNurC+7WscFrfC0h6whbS0BdCCzC+aTsrN9sWThkVP3u33QOy7KYZPfA/Ksiclgcp\\nLLLXYPTHD/EMQDkL4VGLgqjKc+a4GQeEV1JFjBw1sM2v+hAR2d+TcGEjYP9hivk3\\nUg2Sb0aAu02eVRKPExYKqjgyRiVrJEoreM+8KywG8RpfmOTbR/gAQZ6RJTMXrIzi\\nzZRhEwljLiGFcXvVFDBFlqEtdUdBxvVg/GTtgXmoSqZvzXC/v0MCAwEAAaNCMEAw\\nDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFGQVxMKT\\nhr9vfUUHTQkhSoJlXiiyMA0GCSqGSIb3DQEBCwUAA4IBAQCS8400xaT7dAhgKuLu\\nGzNwrqm7JWyCd/MI6G/puDSFKng3zrTCQFkXsCSIxC2FjJbyhYn+J6sXPhmhb0hd\\niWuMvxDS2BqH0O1LfVwy8tsvqLanfSPqphCslUXGZmocpa9ZegjPfQjnHe3vvwlW\\nCBbt8uMlWk3ssWo7YoZyfQlRFcYSkZ1np+HxCHswC8iTWpMEpFu+cEutxp/EE6J0\\nkRfC731wBDK8FDrnDwyp3QXQeRrMuSyssBMu9JLv+s8PZXJNZS7F1HmOwgMQXrHN\\nGHlOR6QnnSWIGMmxFIh28YnTzPrXu6AgOlmrrdYA0g/T3WMKcdXV3RVzBhFFvsBs\\n/0RG\\n-----END CERTIFICATE-----\\n\"}",

May 21 16:23:53 mgmtacm-master-2 installer[4817]: time="2021-05-21T16:23:53Z" level=info msg=" unauthorized: access to the requested resource is not authorized\n" May 21 16:23:53 mgmtacm-master-2 installer[4817]: time="2021-05-21T16:23:53Z" level=info msg="Error: unable to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: unable to pull image: Error initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: (Mirrors also failed: [vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: error pinging docker registry vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com: Get \"https://vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/v2/\": x509: certificate signed by unknown authority]): quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized\n" May 21 16:23:53 mgmtacm-master-2 installer[4817]: time="2021-05-21T16:23:53Z" level=info msg="failed executing nsenter [-t 1 -m -i -- podman run --net host --pid=host --volume /:/rootfs:rw --volume /usr/bin/rpm-ostree:/usr/bin/rpm-ostree --privileged --entrypoint /usr/bin/machine-config-daemon quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 start --node-name localhost --root-mount /rootfs --once-from /opt/install-dir/bootstrap.ign --skip-reboot], env vars [PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=xterm HOSTNAME=mgmtacm-master-2 container=podman HOME=/root], error exit status 125, waitStatus 125, Output \"Error: unable to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: unable to pull image: Error initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: (Mirrors also failed: [vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/ocp4/openshift4@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: error pinging docker registry vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com: Get \"https://vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/v2/\": x509: certificate signed by unknown authority]): quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized\""

pmallan commented 3 years ago

As you see above tried given both server certificate and CA certificate but no help. Local quay server is running on OCP cluster and exposed as route. Have you tested this way in your environment? If so please share some details on the certificate, pull secret to be given in the inventory or AI.

pmallan commented 3 years ago

May 22 13:23:31 mgmtacm-master-1 installer[5860]: time="2021-05-22T13:23:31Z" level=info msg="Writing extracted content to tmp file /opt/extracted_from_ignition.json" May 22 13:23:31 mgmtacm-master-1 installer[5860]: time="2021-05-22T13:23:31Z" level=info msg="Moving /opt/extracted_from_ignition.json to /root/.docker/config.json" May 22 13:23:31 mgmtacm-master-1 installer[5860]: time="2021-05-22T13:23:31Z" level=info msg="Writing image and ignition to disk with arguments: [install --insecure -i /opt/install-dir/master-7052f338-76cd-377c-fc99-91a71ea180dc.ign /dev/sda]" May 22 13:23:31 mgmtacm-master-1 installer[5860]: time="2021-05-22T13:23:31Z" level=info msg="Extracting ignition to disk using quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 mcoImage" May 22 13:23:31 mgmtacm-master-1 installer[5860]: time="2021-05-22T13:23:31Z" level=info msg="Trying to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2...\n" May 22 13:23:31 mgmtacm-master-1 installer[5860]: time="2021-05-22T13:23:31Z" level=info msg="Installing Red Hat Enterprise Linux CoreOS 46.82.202012051820-0 (Ootpa) x86_64 (512-byte sectors)\n" May 22 13:23:31 mgmtacm-master-1 installer[5860]: time="2021-05-22T13:23:31Z" level=info msg=" unauthorized: access to the requested resource is not authorized\n" May 22 13:23:31 mgmtacm-master-1 installer[5860]: time="2021-05-22T13:23:31Z" level=info msg="Error: unable to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: unable to pull image: Error initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: (Mirrors also failed: [mirrorregistry.rtp.raleigh.ibm.com:5000/ocp4/openshift4@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: error pinging docker registry mirrorregistry.rtp.raleigh.ibm.com:5000: Get \"https://mirrorregistry.rtp.raleigh.ibm.com:5000/v2/\": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0]): quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized\n" May 22 13:23:31 mgmtacm-master-1 installer[5860]: time="2021-05-22T13:23:31Z" level=info msg="failed executing nsenter [-t 1 -m -i -- podman run --net host --pid=host --volume /:/rootfs:rw --volume /usr/bin/rpm-ostree:/usr/bin/rpm-ostree --privileged --entrypoint /usr/bin/machine-config-daemon quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 start --node-name localhost --root-mount /rootfs --once-from /opt/install-dir/bootstrap.ign --skip-reboot], env vars [PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=xterm HOSTNAME=mgmtacm-master-1 container=podman HOME=/root], error exit status 125, waitStatus 125, Output \"Error: unable to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: unable to pull image: Error initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: (Mirrors also failed: [mirrorregistry.rtp.raleigh.ibm.com:5000/ocp4/openshift4@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: error pinging docker registry mirrorregistry.rtp.raleigh.ibm.com:5000: Get \"https://mirrorregistry.rtp.raleigh.ibm.com:5000/v2/\": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0]): quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: Error reading manifest sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2 in quay.io/openshift-release-dev/ocp-v4.0-art-dev: unauthorized: access to the requested resource is not authorized\""

pmallan commented 3 years ago

@yrobla I have tried with docker registry running on the RHEL8 VM. The above error occurred. Is there a way to set this environment varaible GODEBUG=x509ignoreCN=0 to avoid this error?

Mirrors also failed: [mirrorregistry.rtp.raleigh.ibm.com:5000/ocp4/openshift4@sha256:5f865a41ea3262fa35bacbfa3f2ffe864ca6681b74ba2868ef13e2120e2080a2: error pinging docker registry mirrorregistry.rtp.raleigh.ibm.com:5000: Get "https://mirrorregistry.rtp.raleigh.ibm.com:5000/v2/": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0]

yrobla commented 3 years ago

You may need to generate your certificate in a different way. Please look at https://stackoverflow.com/questions/64814173/how-do-i-use-sans-with-openssl-instead-of-common-name , or https://geekflare.com/san-ssl-certificate/

pmallan commented 3 years ago

@yrobla yes, I had to create the certificate with SAN (Subject alternative name) in it. I deployed the local registry server with the certificate and I could deploy the cluster in airgap environment.

However there is failure with quay registry running on OCP cluster. The certificate presented by quay is not trusted and throws this error Get "https://vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com/v2/": x509: certificate signed by unknown authority. Do you know location of ca cert (for the certificate used for routes) is stored in cluster

pmallan commented 3 years ago

I could deploy the cluster with local quay registry server as well. Thanks for your help. You can close this issue.

pmallan commented 3 years ago

@yrobla there is a issue with adding worker node in air gap environment. Please find steps we executed

Have installed Control plane cluster using using sonofspike (https://github.com/sonofspike/cluster_mgnt_roles) on 3 bare metal servers in the restricted network mode. We had used sonofspike as ZTP do not deploy cluster on 3 bare metal servers
As I mentioned above, the control plane cluster deployed with local registry
Planning to add worker node this control plane cluster using ztp playbooks. It is dell server. The worker node do not get discovered in the OCP Cluster. AI could discover and initiated request to add the node to the cluster. However nothing happen afterwards

Error: [root@vcuhost ai-deploy-cluster-remoteworker]# aicli info host sos-worker-3 api_vip_connectivity: {"is_success":true} checked_in_at: 2021-05-26T11:49:55.649Z cluster_id: d1c228d2-b565-40a5-84b8-e669331e4ef5 created_at: 2021-05-26T11:43:55.128Z id: 43b35534-49d7-9bde-33e1-23037ebfa7cb installation_disk_path: /dev/sda logs_info: completed logs_started_at: 2021-05-26T11:50:35.232Z machine_config_pool_name: worker ntp_sources: [{"source_name":"ntp3.hetzner.de","source_state":"synced"},{"source_name":"s216-232-132-102.bc.hsia.telus.net","source_state":"combined"},{"source_name":"ntp.xtom.com","source_state":"not_combined"},{"source_name":"sismox.com","source_state":"combined"}] progress: {'current_stage': 'Rebooting', 'stage_started_at': '2021-05-26T11:50:35.579Z', 'stage_updated_at': '2021-05-26T11:50:35.579Z'} requested_hostname: sos-worker-3 role: worker stage_started_at: 0001-01-01T00:00:00.000Z stage_updated_at: 0001-01-01T00:00:00.000Z status: added-to-existing-cluster status_info: Rebooting status_updated_at: 2021-05-26T11:50:35.579Z updated_at: 2021-05-26T11:50:35.584Z user_name: admin

Inventory file: [all:vars]

pull_secret={"auths":{"vran-quay-registry-quay-openshift-operators.apps.mgmtcluster.rtp.raleigh.ibm.com:443":{"auth":"dnJhbmFkbWluOnBhc3N3b3Jk","email":"pmallana@in.ibm.com"}}}

ssh_public_key=ssh-rsa ****

ai_url="http://172.29.231.89:8080" iso_url="http://172.29.231.89"

cluster_name="ocp-cluster-46-3" cluster_domain=rtp.raleigh.ibm.com cluster_version=4.6 cluster_sdn=OVNKubernetes

ingress_vip=172.29.231.239 api_vip=172.29.231.238

bridge_name=ztp-br libvirt_uri="qemu:///system" need_racadm=true

temporary_path=/tmp

final_iso_path=/var/www/html

libvirt_images_path

network_config_path

haproxy_http_port=8008

haproxy_https_port=8443

assisted_tag=stable.17.03.2021-13.18 ai_version="e0df002062f80149769707e72e5952da16897aef"

[provisioner] localhost ansible_connection=local

[master_nodes]

[worker_nodes] sos-worker-3 hostname=sos-worker-3 ip=172.29.231.46 name=sample-worker-1 bmc_type=Dell bmc_address=172.29.230.46 bmc_user=root bmc_password=**** redeploy=false

pmallan commented 3 years ago

It remains in the rebooting state forever Error: [root@vcuhost ai-deploy-cluster-remoteworker]# aicli info host sos-worker-3 api_vip_connectivity: {"is_success":true} checked_in_at: 2021-05-26T11:49:55.649Z cluster_id: d1c228d2-b565-40a5-84b8-e669331e4ef5 created_at: 2021-05-26T11:43:55.128Z id: 43b35534-49d7-9bde-33e1-23037ebfa7cb installation_disk_path: /dev/sda logs_info: completed logs_started_at: 2021-05-26T11:50:35.232Z machine_config_pool_name: worker ntp_sources: [{"source_name":"ntp3.hetzner.de","source_state":"synced"},{"source_name":"s216-232-132-102.bc.hsia.telus.net","source_state":"combined"},{"source_name":"ntp.xtom.com","source_state":"not_combined"},{"source_name":"sismox.com","source_state":"combined"}] progress: {'current_stage': 'Rebooting', 'stage_started_at': '2021-05-26T11:50:35.579Z', 'stage_updated_at': '2021-05-26T11:50:35.579Z'} requested_hostname: sos-worker-3 role: worker stage_started_at: 0001-01-01T00:00:00.000Z stage_updated_at: 0001-01-01T00:00:00.000Z status: added-to-existing-cluster status_info: Rebooting status_updated_at: 2021-05-26T11:50:35.579Z updated_at: 2021-05-26T11:50:35.584Z user_name: admin

pmallan commented 3 years ago

@yrobla there is no option to provide the local registry server details in the ai-deploy-cluster-remoteworker/inventory file. The master node suppose to take care of installing OCP on the worker node right?. The control plane cluster has the details of local registry and secrets.

pmallan commented 3 years ago

sonofspike inventory https://github.com/sonofspike/cluster_mgnt_roles/blob/master/inventory provide these variables to use mirror registry

Parameters for a Restricted Network installation

use_mirror= False mirror_certificate="{{ lookup('file', './mirror_certificate.txt') }}" mirror_registry=infra.ldc-griffintown.rdc-mtl.ran.lan:5000 By using these variables, the control plane cluster has been installed using mirror registry on bare metal servers.

No such option in ZTP inventory https://github.com/redhat-ztp/ztp-cluster-deploy/blob/420f5559ba26d43ce7f2f7d39cb21ff4b5f7cfd2/ai-deploy-cluster-remoteworker/inventory/hosts.sample to provide the mirror registry details. As mentioned above, the worker node do not get added to the cluster. It is hung forever.

yrobla commented 3 years ago

So no, as we commented in previous calls, the work on stable/4.7 is for demo/poc and we are evolving towards the new version with Assisted Installer Operator. There will be support for offline mirroring . However, feel free to send a pull request covering this gap, and we'll review and merge towards stable/4.7

redhat-ztp / ztp-cluster-deploy

[ibm-telco] Playbooks should be supported to run in air gap environment #60

Parameters for a Restricted Network installation

libvirt_images_path

network_config_path

Parameters for a Restricted Network installation