search

openshift / appliance

OpenShift-based Appliance Builder
Apache License 2.0
22 stars 26 forks source link

registry container stuck in "Created" status #267

Open yuchao4x opened 1 month ago

yuchao4x commented 1 month ago

I installed a OCP 4.16.0 on Dell R750 server with appliance, and using the own registry

podman ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 938de0cac8e4 docker.io/library/registry:2 /etc/docker/regis... 18 hours ago Up 18 hours registry

But for some reason, the status of the registry container changed from Up to Created, and then it could not be started automaticly.

I understood that the setup-local-registry service would automatically start it, but it did not. The service log is as follows:

Jul 30 03:03:10 wse-sno143-hongwei systemd[1]: Starting Local Registry... Jul 30 03:03:10 wse-sno143-hongwei setup-local-registry.sh[1522513]: mount: /var/mnt/agentdata: /dev/sdb6 already mounted on /var/mnt/agentdata. Jul 30 03:03:10 wse-sno143-hongwei setup-local-registry.sh[1522514]: Loaded image: docker.io/library/registry:2 Jul 30 03:03:10 wse-sno143-hongwei podman[1522514]: 2024-07-30 03:03:10.503194982 +0000 UTC m=+0.074421228 image loadfromarchive 6a3edb1d5eb6923955548ab7f391e5cc2f04e929c3d32447a7480d8b7a87a0b2 /mnt/agentdata/images/registry/registry.tar Jul 30 03:03:11 wse-sno143-hongwei setup-local-registry.sh[1522540]: .........+...+......+...+..+...+..........+...........+....+..+.........+.+.........+...........+.+..+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+...+....+......+......> Jul 30 03:03:12 wse-sno143-hongwei setup-local-registry.sh[1522540]: ...........+.+...+...........+...+.......+...+.................+.+..+.+......+.....+....+.........+..+......+...+.......+........+....+..+...+...............+......+....+++++++++++++++++++++++++++++++> Jul 30 03:03:12 wse-sno143-hongwei setup-local-registry.sh[1522540]: ----- Jul 30 03:03:13 wse-sno143-hongwei systemd[1]: Started Local Registry. Jul 30 03:03:13 wse-sno143-hongwei podman[1522611]: Port mappings have been discarded as one of the Host, Container, Pod, and None network modes are in use Jul 30 03:03:13 wse-sno143-hongwei podman[1522611]: Error: creating container storage: the container name "registry" is already in use by 8c987959130beb5a07b36022c1dc40e6191c2b5c4ce1eae4020f4a1f09ceff6a. You have to remove that container to be able to reuse that name: > Jul 30 03:03:13 wse-sno143-hongwei podman[1522611]: 2024-07-30 03:03:13.555502797 +0000 UTC m=+0.048872755 image pull 6a3edb1d5eb6923955548ab7f391e5cc2f04e929c3d32447a7480d8b7a87a0b2 docker.io/library/registry:2 Jul 30 03:03:13 wse-sno143-hongwei systemd[1]: start-local-registry.service: Main process exited, code=exited, status=125/n/a Jul 30 03:03:13 wse-sno143-hongwei systemd[1]: start-local-registry.service: Failed with result 'exit-code'.

yuchao4x commented 1 month ago

Several servers have this problem. The current manual solution is to manually delete the container and then start the start-local-registry service.

  1. podman rm -f registry
  2. systemctl start start-local-registry
yuchao4x commented 1 month ago

I also asked appliance team to help me solve this problem fundamentally so that the registry container can always run.

danielerez commented 1 month ago

Hey @yuchao4x. It might be an issue with the registry image, a couple of things to try:

wzhu2x commented 4 weeks ago

[root@8d141 ~]# podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [root@8d141 ~]# [root@8d141 ~]# [root@8d141 ~]# [root@8d141 ~]# [root@8d141 ~]# podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 52e76681e4be quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b6ee79b28e5b577df5d2e78c5d20b367b69a4eb87a6cd831a6c711e24daab251 firstboot-complet... 12 days ago Created beautiful_williamson ac4702dd4d5e docker.io/library/registry:2 /etc/docker/regis... 3 hours ago Created registry

[root@8d141 ~]# systemctl status start-local-registry.service ● start-local-registry.service - Local Registry Loaded: loaded (/etc/systemd/system/start-local-registry.service; enabled; preset: disabled) Active: activating (auto-restart) (Result: exit-code) since Mon 2024-08-19 09:19:34 UTC; 9s ago Process: 355886 ExecStartPre=/bin/rm -f /run/start-local-registry.service.ctr-id (code=exited, status=0/SUCCESS) Process: 355887 ExecStartPre=/usr/local/bin/setup-local-registry.sh (code=exited, status=0/SUCCESS) Process: 356150 ExecStart=podman run --net host --cidfile=/run/start-local-registry.service.ctr-id --privileged --log-driver=journald --name=registry -p 5000:5000 -p 5443:5000 -v ${REGISTRY_DATA}:/var/lib/registry -v /tmp/certs:/certs -e R> Process: 356207 ExecStopPost=/usr/bin/podman rm -f --ignore --cidfile=/run/start-local-registry.service.ctr-id (code=exited, status=0/SUCCESS) Main PID: 356150 (code=exited, status=125) CPU: 2.514s lines 1-9/9 (END)

[root@8d141 ~]# ll /run/start-local-registry.service.ctr-id ls: cannot access '/run/start-local-registry.service.ctr-id': No such file or directory

[root@8d141 ~]# journalctl -f -b -u start-local-registry.service Aug 19 09:20:25 8d141 setup-local-registry.sh[357755]: .+........+...+.+...+......+............+..+...............+............+...+......+..........+..+..........+..+.+......+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++.....+....+..+.+.........+.........+.....+.+...+..+.............+..+....+.........+..+.+..+......................+........+...+....+.....+.+...+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++..+...+.......+.................+.+.............................+.+.....................+.....+...+.....................+....+.....+...+....+........+......................+......+..............+...+.+......+...............+...........+.........+...+.+......+.........+..+.........................+........+.......+..+.+.....+......+...+.......+.....+.+............+..............................+...+...+..+............................+.....+.+...+..+..........+.........+.................+....+...+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Aug 19 09:20:29 8d141 setup-local-registry.sh[357755]: ...+.....+.+.....+.+........+...+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++..+.....+.+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++.......+.............+.........+..+....+...........+..........+........+...+.......+...+............+......+..+.........+......+....+.........+......+...+..+...+......+......+....+..............................+.....+....+..+..........+.....+.........+.......+...+..+...+......+.+..................+......+...........+.......+..+....+..+............+.......+.....+....+.....+.+......+..+...+...+....+...........+.........+...+....+.....................+.....+.......+...+............+..+....+..+.............+.........+.....+.+.................+.......+.................+.........................+.....+...+...+....+...+...+.................+.+.....+......+..........+........+.......+......+...+...........+.............+...............+.....+......+.......+........+.......+..............+.........+.+..............+....+..+...+..........+...............+.....+.+..+...+.+......+..+............+.+..+............+......+.........+...............+.+.................+...+...+.........+...+....+...+...+........+......+.+........+....+...............+..+.........+.......+...+..+....+.........+.....+............+...+.+...............+.....+.+........+.+......+..+......+.+.......................................+....................+.........+.............+..+.......+.....+.+..............................+...+......+...........+...+.+...+...............+..+.+........+......................+.....+...+....+......+.....+....+........+....+.........+..+..........+...+...........+....+....................+......+.+.....+...............+..........+...+...............+..+....+...+.....+......+.........+.......+.....+............................+..+.+...+..+.........+.............+..+......+............+..........+...+........+.+........+...+...+.+......+........+............+.......+..+.+..+.+.........+..+...+...+......................+......+..............+.+..............+......+.+..............+..........+..+....+..............+.......+.......................+.......+.........+.....+......+....+.....+...............+.+...+..+.......+.....+.............+...+...........+.....................+...+......+................+...+.....+....+.....+.+.........+...........+.......+...+......+.....................+...+.....+....+.....+.....................+.......+............+......+.....+.+..............+.+...+..+....+..............+...+..........+.........+.........+.....+................+..............+.+...+......+......+..............+.+.....+.............+...............+......+..+...+.......+.....+.........+................+..+.+......+......+..+............+...+.............+........+......+......+...+....+.......................+....+........+..........+..+...+.+...........+.+...+.........+........+...+......+.............+.....+......................+...+......+..+..................+.......+.................+.....................+.........+.+..+.........+......+....+...+...............+...+...+.....................+.....+...................+..+...+.........+...+.............+.........+........+...+.......+..............+.+.........+.....+......+....+....................+...+.+......+...+.....+.+..+....+...........+.............+......+...............+.....+...+.......+.....+..................+...............+.+..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Aug 19 09:20:29 8d141 setup-local-registry.sh[357755]: ----- Aug 19 09:20:30 8d141 systemd[1]: Started Local Registry. Aug 19 09:20:30 8d141 podman[357913]: Port mappings have been discarded as one of the Host, Container, Pod, and None network modes are in use Aug 19 09:20:30 8d141 podman[357913]: Error: creating container storage: the container name "registry" is already in use by ac4702dd4d5e345a6bd4c0a2b5047ee57141f36e29377820ef02aa2a51f731c5. You have to remove that container to be able to reuse that name: that name is already in use, or use --replace to instruct Podman to do so. Aug 19 09:20:30 8d141 podman[357913]: 2024-08-19 09:20:30.077007879 +0000 UTC m=+0.045686829 image pull f2ed136cbeabb394957a9c4652cfd7f0098593f65836dfca413287fd1de36568 docker.io/library/registry:2 Aug 19 09:20:30 8d141 systemd[1]: start-local-registry.service: Main process exited, code=exited, status=125/n/a Aug 19 09:20:30 8d141 systemd[1]: start-local-registry.service: Failed with result 'exit-code'. Aug 19 09:20:30 8d141 systemd[1]: start-local-registry.service: Consumed 5.478s CPU time. Aug 19 09:20:40 8d141 systemd[1]: start-local-registry.service: Scheduled restart job, restart counter is at 699. Aug 19 09:20:40 8d141 systemd[1]: Stopped Local Registry. Aug 19 09:20:40 8d141 systemd[1]: start-local-registry.service: Consumed 5.478s CPU time. Aug 19 09:20:40 8d141 systemd[1]: Starting Local Registry... Aug 19 09:20:40 8d141 setup-local-registry.sh[358616]: mount: /var/mnt/agentdata: /dev/sdb6 already mounted on /var/mnt/agentdata. Aug 19 09:20:40 8d141 setup-local-registry.sh[358617]: Loaded image: docker.io/library/registry:2

========= hello danielerez i also meet this issue here is some logs

danielerez commented 4 weeks ago

[root@8d141 ~]# podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [root@8d141 ~]# [root@8d141 ~]# [root@8d141 ~]# [root@8d141 ~]# [root@8d141 ~]# podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 52e76681e4be quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b6ee79b28e5b577df5d2e78c5d20b367b69a4eb87a6cd831a6c711e24daab251 firstboot-complet... 12 days ago Created beautiful_williamson ac4702dd4d5e docker.io/library/registry:2 /etc/docker/regis... 3 hours ago Created registry

[root@8d141 ~]# systemctl status start-local-registry.service ● start-local-registry.service - Local Registry Loaded: loaded (/etc/systemd/system/start-local-registry.service; enabled; preset: disabled) Active: activating (auto-restart) (Result: exit-code) since Mon 2024-08-19 09:19:34 UTC; 9s ago Process: 355886 ExecStartPre=/bin/rm -f /run/start-local-registry.service.ctr-id (code=exited, status=0/SUCCESS) Process: 355887 ExecStartPre=/usr/local/bin/setup-local-registry.sh (code=exited, status=0/SUCCESS) Process: 356150 ExecStart=podman run --net host --cidfile=/run/start-local-registry.service.ctr-id --privileged --log-driver=journald --name=registry -p 5000:5000 -p 5443:5000 -v ${REGISTRY_DATA}:/var/lib/registry -v /tmp/certs:/certs -e R> Process: 356207 ExecStopPost=/usr/bin/podman rm -f --ignore --cidfile=/run/start-local-registry.service.ctr-id (code=exited, status=0/SUCCESS) Main PID: 356150 (code=exited, status=125) CPU: 2.514s lines 1-9/9 (END)

[root@8d141 ~]# ll /run/start-local-registry.service.ctr-id ls: cannot access '/run/start-local-registry.service.ctr-id': No such file or directory

[root@8d141 ~]# journalctl -f -b -u start-local-registry.service Aug 19 09:20:25 8d141 setup-local-registry.sh[357755]: .+........+...+.+...+......+............+..+...............+............+...+......+..........+..+..........+..+.+......+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++.....+....+..+.+.........+.........+.....+.+...+..+.............+..+....+.........+..+.+..+......................+........+...+....+.....+.+...+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++..+...+.......+.................+.+.............................+.+.....................+.....+...+.....................+....+.....+...+....+........+......................+......+..............+...+.+......+...............+...........+.........+...+.+......+.........+..+.........................+........+.......+..+.+.....+......+...+.......+.....+.+............+..............................+...+...+..+............................+.....+.+...+..+..........+.........+.................+....+...+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Aug 19 09:20:29 8d141 setup-local-registry.sh[357755]: ...+.....+.+.....+.+........+...+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++..+.....+.+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++.......+.............+.........+..+....+...........+..........+........+...+.......+...+............+......+..+.........+......+....+.........+......+...+..+...+......+......+....+..............................+.....+....+..+..........+.....+.........+.......+...+..+...+......+.+..................+......+...........+.......+..+....+..+............+.......+.....+....+.....+.+......+..+...+...+....+...........+.........+...+....+.....................+.....+.......+...+............+..+....+..+.............+.........+.....+.+.................+.......+.................+.........................+.....+...+...+....+...+...+.................+.+.....+......+..........+........+.......+......+...+...........+.............+...............+.....+......+.......+........+.......+..............+.........+.+..............+....+..+...+..........+...............+.....+.+..+...+.+......+..+............+.+..+............+......+.........+...............+.+.................+...+...+.........+...+....+...+...+........+......+.+........+....+...............+..+.........+.......+...+..+....+.........+.....+............+...+.+...............+.....+.+........+.+......+..+......+.+.......................................+....................+.........+.............+..+.......+.....+.+..............................+...+......+...........+...+.+...+...............+..+.+........+......................+.....+...+....+......+.....+....+........+....+.........+..+..........+...+...........+....+....................+......+.+.....+...............+..........+...+...............+..+....+...+.....+......+.........+.......+.....+............................+..+.+...+..+.........+.............+..+......+............+..........+...+........+.+........+...+...+.+......+........+............+.......+..+.+..+.+.........+..+...+...+......................+......+..............+.+..............+......+.+..............+..........+..+....+..............+.......+.......................+.......+.........+.....+......+....+.....+...............+.+...+..+.......+.....+.............+...+...........+.....................+...+......+................+...+.....+....+.....+.+.........+...........+.......+...+......+.....................+...+.....+....+.....+.....................+.......+............+......+.....+.+..............+.+...+..+....+..............+...+..........+.........+.........+.....+................+..............+.+...+......+......+..............+.+.....+.............+...............+......+..+...+.......+.....+.........+................+..+.+......+......+..+............+...+.............+........+......+......+...+....+.......................+....+........+..........+..+...+.+...........+.+...+.........+........+...+......+.............+.....+......................+...+......+..+..................+.......+.................+.....................+.........+.+..+.........+......+....+...+...............+...+...+.....................+.....+...................+..+...+.........+...+.............+.........+........+...+.......+..............+.+.........+.....+......+....+....................+...+.+......+...+.....+.+..+....+...........+.............+......+...............+.....+...+.......+.....+..................+...............+.+..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Aug 19 09:20:29 8d141 setup-local-registry.sh[357755]: ----- Aug 19 09:20:30 8d141 systemd[1]: Started Local Registry. Aug 19 09:20:30 8d141 podman[357913]: Port mappings have been discarded as one of the Host, Container, Pod, and None network modes are in use Aug 19 09:20:30 8d141 podman[357913]: Error: creating container storage: the container name "registry" is already in use by ac4702dd4d5e345a6bd4c0a2b5047ee57141f36e29377820ef02aa2a51f731c5. You have to remove that container to be able to reuse that name: that name is already in use, or use --replace to instruct Podman to do so. Aug 19 09:20:30 8d141 podman[357913]: 2024-08-19 09:20:30.077007879 +0000 UTC m=+0.045686829 image pull f2ed136cbeabb394957a9c4652cfd7f0098593f65836dfca413287fd1de36568 docker.io/library/registry:2 Aug 19 09:20:30 8d141 systemd[1]: start-local-registry.service: Main process exited, code=exited, status=125/n/a Aug 19 09:20:30 8d141 systemd[1]: start-local-registry.service: Failed with result 'exit-code'. Aug 19 09:20:30 8d141 systemd[1]: start-local-registry.service: Consumed 5.478s CPU time. Aug 19 09:20:40 8d141 systemd[1]: start-local-registry.service: Scheduled restart job, restart counter is at 699. Aug 19 09:20:40 8d141 systemd[1]: Stopped Local Registry. Aug 19 09:20:40 8d141 systemd[1]: start-local-registry.service: Consumed 5.478s CPU time. Aug 19 09:20:40 8d141 systemd[1]: Starting Local Registry... Aug 19 09:20:40 8d141 setup-local-registry.sh[358616]: mount: /var/mnt/agentdata: /dev/sdb6 already mounted on /var/mnt/agentdata. Aug 19 09:20:40 8d141 setup-local-registry.sh[358617]: Loaded image: docker.io/library/registry:2

========= hello danielerez i also meet this issue here is some logs

Thanks @wzhu2x. Can you please also attach the logs of the existing pod?

podman logs ac4702dd4d5e
wzhu2x commented 3 weeks ago

[root@sno paas]# podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [root@sno paas]# podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9a315159983f docker.io/library/registry:2 /etc/docker/regis... 10 hours ago Created registry [root@sno paas]# podman logs 9a315159983f time="2024-08-19T19:10:14.787117496Z" level=warning msg="No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable." go.version=go1.20.8 instance.id=9112a01c-d28b-44a4-adcb-55ed44ac95fa service=registry version=2.8.3 time="2024-08-19T19:10:14.787733318Z" level=info msg="redis not configured" go.version=go1.20.8 instance.id=9112a01c-d28b-44a4-adcb-55ed44ac95fa service=registry version=2.8.3 time="2024-08-19T19:10:14.78773567Z" level=info msg="Starting upload purge in 25m0s" go.version=go1.20.8 instance.id=9112a01c-d28b-44a4-adcb-55ed44ac95fa service=registry version=2.8.3 time="2024-08-19T19:10:14.787842801Z" level=info msg="using inmemory blob descriptor cache" go.version=go1.20.8 instance.id=9112a01c-d28b-44a4-adcb-55ed44ac95fa service=registry version=2.8.3 time="2024-08-19T19:10:14.800556123Z" level=info msg="restricting TLS version to tls1.2 or higher" go.version=go1.20.8 instance.id=9112a01c-d28b-44a4-adcb-55ed44ac95fa service=registry version=2.8.3 time="2024-08-19T19:10:14.801446839Z" level=info msg="restricting TLS cipher suites to: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_AES_128_GCM_SHA256,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_256_GCM_SHA384" go.version=go1.20.8 instance.id=9112a01c-d28b-44a4-adcb-55ed44ac95fa service=registry version=2.8.3 time="2024-08-19T19:10:14.806185859Z" level=info msg="listening on [::]:5000, tls" go.version=go1.20.8 instance.id=9112a01c-d28b-44a4-adcb-55ed44ac95fa service=registry version=2.8.3 time="2024-08-19T19:11:53.934352107Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=4ece1439-b3de-485e-9996-be724802def8 http.request.method=GET http.request.remoteaddr="127.0.0.1:56148" http.request.uri="/v2/" http.request.useragent="containers/5.29.0 (github.com/containers/image)" http.response.contenttype="application/json; charset=utf-8" http.response.duration="887.36µs" http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:11:53 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "containers/5.29.0 (github.com/containers/image)" time="2024-08-19T19:11:53.960052069Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=d51ce5f8-30f7-4373-88a6-f28033b8449f http.request.method=GET http.request.remoteaddr="127.0.0.1:56164" http.request.uri="/v2/openshift/release/manifests/sha256:eaa7835f2ec7d2513a76e30a41c21ce62ec11313fab2f8f3f46dd4999957a883" http.request.useragent="containers/5.29.0 (github.com/containers/image)" http.response.contenttype="application/vnd.docker.distribution.manifest.v2+json" http.response.duration=8.241786ms http.response.status=200 http.response.written=8647 127.0.0.1 - - [19/Aug/2024:19:11:53 +0000] "GET /v2/openshift/release/manifests/sha256:eaa7835f2ec7d2513a76e30a41c21ce62ec11313fab2f8f3f46dd4999957a883 HTTP/1.1" 200 8647 "" "containers/5.29.0 (github.com/containers/image)" time="2024-08-19T19:11:53.965344481Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=33225992-d297-4dd3-8c9e-b18c9bcf77a6 http.request.method=GET http.request.remoteaddr="127.0.0.1:56164" http.request.uri="/v2/openshift/release/blobs/sha256:875d12fac2731b959154beb6ae0c80ea49d3a15a533d6586f4e38aa999ac5ece" http.request.useragent="containers/5.29.0 (github.com/containers/image)" http.response.contenttype="application/octet-stream" http.response.duration=4.568382ms http.response.status=200 http.response.written=9908 127.0.0.1 - - [19/Aug/2024:19:11:53 +0000] "GET /v2/openshift/release/blobs/sha256:875d12fac2731b959154beb6ae0c80ea49d3a15a533d6586f4e38aa999ac5ece HTTP/1.1" 200 9908 "" "containers/5.29.0 (github.com/containers/image)" time="2024-08-19T19:12:05.480801764Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=be10ac5e-1886-4d06-a0fb-95af972d957f http.request.method=GET http.request.remoteaddr="127.0.0.1:49062" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="741.382µs" http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:12:05 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:05.501158467Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=693e3d08-8b4f-4ad5-85f6-05dffe9bcee9 http.request.method=GET http.request.remoteaddr="127.0.0.1:49076" http.request.uri="/v2/redhat/redhat-operator-index/manifests/v4.16" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/vnd.docker.distribution.manifest.list.v2+json" http.response.duration=3.127551ms http.response.status=200 http.response.written=959 127.0.0.1 - - [19/Aug/2024:19:12:05 +0000] "GET /v2/redhat/redhat-operator-index/manifests/v4.16 HTTP/1.1" 200 959 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:05.503078503Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=4c9b752d-f0c0-4f0b-b3b8-4f68767f908a http.request.method=GET http.request.remoteaddr="127.0.0.1:49076" http.request.uri="/v2/redhat/redhat-operator-index/manifests/sha256:c372411eb8d941d3c76fd445cd7b72149f9ee9f387ef48990d41a87cdecffd79" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/vnd.docker.distribution.manifest.v2+json" http.response.duration=1.369592ms http.response.status=200 http.response.written=1092 127.0.0.1 - - [19/Aug/2024:19:12:05 +0000] "GET /v2/redhat/redhat-operator-index/manifests/sha256:c372411eb8d941d3c76fd445cd7b72149f9ee9f387ef48990d41a87cdecffd79 HTTP/1.1" 200 1092 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.335430576Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=75e4bd22-a19a-48b0-a7fd-be59c7fadbfb http.request.method=GET http.request.remoteaddr="127.0.0.1:49084" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.001073ms http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.353229962Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=ca73589e-9d95-4088-bd00-373faa8b46bb http.request.method=GET http.request.remoteaddr="127.0.0.1:49096" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="989.575µs" http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.372309623Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=7318e9a9-87bd-4aea-808a-8828d925bfbc http.request.method=GET http.request.remoteaddr="127.0.0.1:49102" http.request.uri="/v2/redhat/certified-operator-index/manifests/v4.16" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/vnd.docker.distribution.manifest.list.v2+json" http.response.duration=1.680108ms http.response.status=200 http.response.written=959 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/redhat/certified-operator-index/manifests/v4.16 HTTP/1.1" 200 959 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.372425307Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=dcdaff50-445b-4602-bde8-320f81f690cf http.request.method=GET http.request.remoteaddr="127.0.0.1:49100" http.request.uri="/v2/redhat/certified-operator-index/manifests/v4.16" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/vnd.docker.distribution.manifest.list.v2+json" http.response.duration=3.347998ms http.response.status=200 http.response.written=959 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/redhat/certified-operator-index/manifests/v4.16 HTTP/1.1" 200 959 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.375103824Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=bc741743-f0d0-4f8b-8000-f01cf091c08a http.request.method=GET http.request.remoteaddr="127.0.0.1:49100" http.request.uri="/v2/redhat/certified-operator-index/manifests/sha256:70fe63d3a68d56d9937f7ff1d49b256bdc080aad2bedc958e46150b3da005349" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/vnd.docker.distribution.manifest.v2+json" http.response.duration=2.308151ms http.response.status=200 http.response.written=1091 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/redhat/certified-operator-index/manifests/sha256:70fe63d3a68d56d9937f7ff1d49b256bdc080aad2bedc958e46150b3da005349 HTTP/1.1" 200 1091 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.375179067Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=f57f8e00-9733-4ad1-9519-9981a0839cb7 http.request.method=GET http.request.remoteaddr="127.0.0.1:49102" http.request.uri="/v2/redhat/certified-operator-index/manifests/sha256:70fe63d3a68d56d9937f7ff1d49b256bdc080aad2bedc958e46150b3da005349" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/vnd.docker.distribution.manifest.v2+json" http.response.duration=2.502108ms http.response.status=200 http.response.written=1091 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/redhat/certified-operator-index/manifests/sha256:70fe63d3a68d56d9937f7ff1d49b256bdc080aad2bedc958e46150b3da005349 HTTP/1.1" 200 1091 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.63520029Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=38b467a4-d536-4b8b-8756-78771a2eef6b http.request.method=GET http.request.remoteaddr="127.0.0.1:49110" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="815.564µs" http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.693844404Z" level=error msg="response completed with error" err.code="manifest unknown" err.detail="unknown tag=latest" err.message="manifest unknown" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=a0b75e8e-e5ec-4eef-9207-28c11409c909 http.request.method=GET http.request.remoteaddr="127.0.0.1:49118" http.request.uri="/v2/ipws/library/busybox/manifests/latest" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=38.986874ms http.response.status=404 http.response.written=96 vars.name="ipws/library/busybox" vars.reference=latest 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/ipws/library/busybox/manifests/latest HTTP/1.1" 404 96 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.838167465Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=1cca85e4-52db-4483-9e43-06c4d5adcf2b http.request.method=GET http.request.remoteaddr="127.0.0.1:49134" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="757.103µs" http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.857171731Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=12660382-1dfe-4778-ba9b-cb5ddcfabcf4 http.request.method=GET http.request.remoteaddr="127.0.0.1:49140" http.request.uri="/v2/redhat/redhat-operator-index/manifests/v4.16" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/vnd.docker.distribution.manifest.list.v2+json" http.response.duration=1.051369ms http.response.status=200 http.response.written=959 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/redhat/redhat-operator-index/manifests/v4.16 HTTP/1.1" 200 959 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:08.858375059Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=03336008-71e9-4983-8c59-ae8f1456c2c8 http.request.method=GET http.request.remoteaddr="127.0.0.1:49140" http.request.uri="/v2/redhat/redhat-operator-index/manifests/sha256:c372411eb8d941d3c76fd445cd7b72149f9ee9f387ef48990d41a87cdecffd79" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/vnd.docker.distribution.manifest.v2+json" http.response.duration="865.844µs" http.response.status=200 http.response.written=1092 127.0.0.1 - - [19/Aug/2024:19:12:08 +0000] "GET /v2/redhat/redhat-operator-index/manifests/sha256:c372411eb8d941d3c76fd445cd7b72149f9ee9f387ef48990d41a87cdecffd79 HTTP/1.1" 200 1092 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:41.990297297Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=0e8e9df4-e01f-45c3-9c55-c57129a6242e http.request.method=GET http.request.remoteaddr="127.0.0.1:50428" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.47671ms http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:12:41 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:12:42.009199725Z" level=error msg="response completed with error" err.code="manifest unknown" err.detail="unknown tag=latest" err.message="manifest unknown" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=7b5ea815-ec66-4136-8db7-ade7e0decbcb http.request.method=GET http.request.remoteaddr="127.0.0.1:50442" http.request.uri="/v2/ipws/library/busybox/manifests/latest" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="840.581µs" http.response.status=404 http.response.written=96 vars.name="ipws/library/busybox" vars.reference=latest 127.0.0.1 - - [19/Aug/2024:19:12:42 +0000] "GET /v2/ipws/library/busybox/manifests/latest HTTP/1.1" 404 96 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:13:24.984158257Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=dbaa3abc-9800-4cca-a779-dd3fd594c506 http.request.method=GET http.request.remoteaddr="127.0.0.1:56072" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="799.74µs" http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:13:24 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:13:25.00299168Z" level=error msg="response completed with error" err.code="manifest unknown" err.detail="unknown tag=latest" err.message="manifest unknown" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=090e62f8-6b66-4693-89b8-99286c2a1146 http.request.method=GET http.request.remoteaddr="127.0.0.1:56086" http.request.uri="/v2/ipws/library/busybox/manifests/latest" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="875.166µs" http.response.status=404 http.response.written=96 vars.name="ipws/library/busybox" vars.reference=latest 127.0.0.1 - - [19/Aug/2024:19:13:25 +0000] "GET /v2/ipws/library/busybox/manifests/latest HTTP/1.1" 404 96 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:14:31.989610123Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=964f34ae-cb06-4db2-b3fd-46298130faeb http.request.method=GET http.request.remoteaddr="127.0.0.1:58586" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="693.604µs" http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:14:31 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:14:32.008755944Z" level=error msg="response completed with error" err.code="manifest unknown" err.detail="unknown tag=latest" err.message="manifest unknown" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=24df4da8-614f-4f89-939b-a87f3085ffff http.request.method=GET http.request.remoteaddr="127.0.0.1:58600" http.request.uri="/v2/ipws/library/busybox/manifests/latest" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.075185ms http.response.status=404 http.response.written=96 vars.name="ipws/library/busybox" vars.reference=latest 127.0.0.1 - - [19/Aug/2024:19:14:32 +0000] "GET /v2/ipws/library/busybox/manifests/latest HTTP/1.1" 404 96 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:16:20.991738467Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=0fe29632-7b04-455e-84ba-0bac396ce5ac http.request.method=GET http.request.remoteaddr="127.0.0.1:54078" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="786.626µs" http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:16:20 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:16:21.009631739Z" level=error msg="response completed with error" err.code="manifest unknown" err.detail="unknown tag=latest" err.message="manifest unknown" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=aaccd195-f26a-44df-9bc0-73eb47db407e http.request.method=GET http.request.remoteaddr="127.0.0.1:54080" http.request.uri="/v2/ipws/library/busybox/manifests/latest" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="820.437µs" http.response.status=404 http.response.written=96 vars.name="ipws/library/busybox" vars.reference=latest 127.0.0.1 - - [19/Aug/2024:19:16:21 +0000] "GET /v2/ipws/library/busybox/manifests/latest HTTP/1.1" 404 96 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:19:20.984128471Z" level=info msg="response completed" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=5cca2283-0d12-4ce9-9c36-e649c259ae20 http.request.method=GET http.request.remoteaddr="127.0.0.1:46700" http.request.uri="/v2/" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration="869.745µs" http.response.status=200 http.response.written=2 127.0.0.1 - - [19/Aug/2024:19:19:20 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" time="2024-08-19T19:19:21.002313734Z" level=error msg="response completed with error" err.code="manifest unknown" err.detail="unknown tag=latest" err.message="manifest unknown" go.version=go1.20.8 http.request.host="registry.appliance.com:5000" http.request.id=7335f856-5b86-4087-8ab5-747d87ea3d10 http.request.method=GET http.request.remoteaddr="127.0.0.1:46702" http.request.uri="/v2/ipws/library/busybox/manifests/latest" http.request.useragent="cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.058327ms http.response.status=404 http.response.written=96 vars.name="ipws/library/busybox" vars.reference=latest 127.0.0.1 - - [19/Aug/2024:19:19:21 +0000] "GET /v2/ipws/library/busybox/manifests/latest HTTP/1.1" 404 96 "" "cri-o/1.29.5-5.rhaos4.16.git7032128.el9 os/linux arch/amd64"