search

crc-org / crc

CRC is a tool to help you run containers. It manages a local OpenShift 4.x cluster, Microshift or a Podman VM optimized for testing and development purposes
https://crc.dev
Apache License 2.0
1.25k stars 236 forks source link

[BUG]OperatorHub is blank can't install any Operators #3761

Closed santoshnt1 closed 1 year ago

santoshnt1 commented 1 year ago

General information

CRC version

# Put `crc version` output here - 2.23.0+ddcfe8

CRC status

# Put `crc status --log-level debug` output here
DEBU CRC version: 2.23.0+ddcfe8
DEBU OpenShift version: 4.13.3
DEBU Podman version: 4.4.4
DEBU Running 'crc status'
CRC VM:          Running
OpenShift:       Running (v4.13.3)
RAM Usage:       7.14GB of 9.369GB
Disk Usage:      15.79GB of 32.68GB (Inside the CRC VM)
Cache Usage:     21.32GB
Cache Directory: C:\Users\skumar2382\.crc\cache
DEBU Running '(Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion" -Name ReleaseId).ReleaseId'
DEBU Running '(Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion").EditionID'

CRC config

# Put `crc config view` output here
- consent-telemetry                     : yes

## Host Operating System
```bash
# Put the output of `cat /etc/os-release` in case of Linux
# put the output of `sw_vers` in case of Mac
# Put the output of `systeminfo` in case of Windows
OS Name:                   Microsoft Windows 10 Enterprise
OS Version:                10.0.19045 N/A Build 19045
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:          
Registered Organization: 
Product ID:                00330-80000-00000-AA761
Original Install Date:     05/07/2023, 17:21:39
System Boot Time:          18/07/2023, 12:33:08
System Manufacturer:       HP
System Model:              HP ZBook Firefly 16 inch G9 Mobile Workstation PC
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 154 Stepping 3 GenuineIntel ~1800 Mhz
BIOS Version:              HP U71 Ver. 01.07.20, 05/06/2023
Windows Directory:         C:\windows
System Directory:          C:\windows\system32
Boot Device:               \Device\HarddiskVolume1
System Locale:             en-us;English (United States)
Input Locale:              en-gb;English (United Kingdom)
Time Zone:                 (UTC+00:00) Dublin, Edinburgh, Lisbon, London
Total Physical Memory:     32,400 MB
Available Physical Memory: 11,591 MB
Virtual Memory: Max Size:  37,264 MB
Virtual Memory: Available: 13,174 MB
Virtual Memory: In Use:    24,090 MB
Page File Location(s):     C:\pagefile.sys
Domain:                    WORKGROUP
Logon Server:             
Hotfix(s):                 8 Hotfix(s) Installed.
                           [01]: KB5027122
                           [02]: KB5012170
                           [03]: KB5015684
                           [04]: KB5026037
                           [05]: KB5027215
                           [06]: KB5014032
                           [07]: KB5020372
                           [08]: KB5026879
Network Card(s):           10 NIC(s) Installed.
                           [01]: Intel(R) Wi-Fi 6E AX211 160MHz
                                 Connection Name: Wi-Fi
                                 Status:          Media disconnected
                           [02]: Bluetooth Device (Personal Area Network)
                                 Connection Name: Bluetooth Network Connection
                                 Status:          Media disconnected
                           [03]: Lenovo USB Ethernet
                                 Connection Name: Ethernet 2
                                 DHCP Enabled:    Yes
                                 DHCP Server:     192.168.0.1
                                 IP address(es)
                                 [01]: 192.168.0.18
                                 [02]: fe80::2eb:fe4f:af21:a2e6
                                 [03]: fdd4:9df2:d7be:0:8905:7393:e7b9:5ba0
                                 [04]: fdd4:9df2:d7be:0:a496:d1b:ecc0:9f20
                           [04]: VirtualBox Host-Only Ethernet Adapter
                                 Connection Name: Ethernet 3
                                 DHCP Enabled:    No
                                 IP address(es)
                                 [01]: 192.168.0.1
                                 [02]: fe80::b4ab:ff64:153:62a4
                           [05]: VMware Virtual Ethernet Adapter for VMnet1
                                 Connection Name: VMware Network Adapter VMnet1
                                 DHCP Enabled:    Yes
                                 DHCP Server:     192.168.254.254
                                 IP address(es)
                                 [01]: 192.168.254.1
                                 [02]: fe80::d25e:6aea:1780:927a
                           [06]: VMware Virtual Ethernet Adapter for VMnet8
                                 Connection Name: VMware Network Adapter VMnet8
                                 DHCP Enabled:    Yes
                                 DHCP Server:     192.168.79.254
                                 IP address(es)
                                 [01]: 192.168.79.1
                                 [02]: fe80::a64d:e29d:6a39:6264
                           [07]: Zscaler Network Adapter 1.0.2.0
                                 Connection Name: Ethernet 4
                                 Status:          Media disconnected
                           [08]: Hyper-V Virtual Ethernet Adapter
                                 Connection Name: vEthernet (Default Switch)
                                 DHCP Enabled:    No
                                 IP address(es)
                                 [01]: 172.18.208.1
                                 [02]: fe80::9d0e:679b:aadb:bfaf
                           [09]: Wintun Userspace Tunnel
                                 Connection Name: Appgate SDP
                                 DHCP Enabled:    No
                                 IP address(es)
                                 [01]: 172.24.255.60
                           [10]: Hyper-V Virtual Ethernet Adapter
                                 Connection Name: vEthernet (WSL)
                                 DHCP Enabled:    No
                                 IP address(es)
                                 [01]: 172.22.16.1
                                 [02]: fe80::6507:8ea8:5330:2e8b
Hyper-V Requirements:      A hypervisor has been detected. Features required for Hyper-V will not be displayed.

### Steps to reproduce

  1. Click on OperatorHub
  2. Check Node
  3. 
  4. 

### Expected
See operators under operatorHUB

### Actual
No Operator inside OperatorHub

### Logs
ime="2023-07-18T13:54:21+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:23+01:00" level=debug msg="retry loop: attempt 1"
time="2023-07-18T13:54:23+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:23+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:23+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:25+01:00" level=debug msg="retry loop: attempt 2"
time="2023-07-18T13:54:25+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:25+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:25+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:27+01:00" level=debug msg="retry loop: attempt 3"
time="2023-07-18T13:54:27+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:27+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:27+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:29+01:00" level=debug msg="retry loop: attempt 4"
time="2023-07-18T13:54:29+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:29+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:29+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:31+01:00" level=debug msg="retry loop: attempt 5"
time="2023-07-18T13:54:31+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:31+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:31+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:33+01:00" level=debug msg="retry loop: attempt 6"
time="2023-07-18T13:54:33+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:33+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:33+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:35+01:00" level=debug msg="retry loop: attempt 7"
time="2023-07-18T13:54:35+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:35+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:35+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:37+01:00" level=debug msg="retry loop: attempt 8"
time="2023-07-18T13:54:37+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:37+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:37+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:39+01:00" level=debug msg="retry loop: attempt 9"
time="2023-07-18T13:54:39+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:40+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:40+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:42+01:00" level=debug msg="retry loop: attempt 10"
time="2023-07-18T13:54:42+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:42+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:42+01:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2023-07-18T13:54:44+01:00" level=debug msg="retry loop: attempt 11"
time="2023-07-18T13:54:44+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:44+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:44+01:00" level=debug msg="Waiting for availability of resource type 'secret'"
time="2023-07-18T13:54:44+01:00" level=debug msg="retry loop: attempt 0"
time="2023-07-18T13:54:44+01:00" level=debug msg="Running SSH command: timeout 5s oc get secret --context admin --cluster crc --kubeconfig /opt/kubeconfig"
time="2023-07-18T13:54:44+01:00" level=debug msg="SSH command results: err: <nil>, output: NAME                       TYPE                                  DATA   AGE\nbuilder-dockercfg-bkmg6    kubernetes.io/dockercfg               1      28d\nbuilder-token-nxfbd        kubernetes.io/service-account-token   4      28d\ndefault-dockercfg-jh8r5    kubernetes.io/dockercfg               1      28d\ndefault-token-rcslw        kubernetes.io/service-account-token   4      28d\ndeployer-dockercfg-lkssj   kubernetes.io/dockercfg               1      28d\ndeployer-token-dczbw       kubernetes.io/service-account-token   4      28d\n"
time="2023-07-18T13:54:44+01:00" level=debug msg="NAME                       TYPE                                  DATA   AGE\nbuilder-dockercfg-bkmg6    kubernetes.io/dockercfg               1      28d\nbuilder-token-nxfbd        kubernetes.io/service-account-token   4      28d\ndefault-dockercfg-jh8r5    kubernetes.io/dockercfg               1      28d\ndefault-token-rcslw        kubernetes.io/service-account-token   4      28d\ndeployer-dockercfg-lkssj   kubernetes.io/dockercfg               1      28d\ndeployer-token-dczbw       kubernetes.io/service-account-token   4      28d\n"
time="2023-07-18T13:54:44+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:44+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:44+01:00" level=info msg="Changing the password for the kubeadmin user"
time="2023-07-18T13:54:44+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:44+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:44+01:00" level=debug msg="Waiting for availability of resource type 'clusterversion'"
time="2023-07-18T13:54:44+01:00" level=debug msg="retry loop: attempt 0"
time="2023-07-18T13:54:44+01:00" level=debug msg="Running SSH command: timeout 5s oc get clusterversion --context admin --cluster crc --kubeconfig /opt/kubeconfig"
time="2023-07-18T13:54:44+01:00" level=debug msg="SSH command results: err: <nil>, output: NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS\nversion   4.13.3    True        False         28d     Cluster version is 4.13.3\n"
time="2023-07-18T13:54:44+01:00" level=debug msg="NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS\nversion   4.13.3    True        False         28d     Cluster version is 4.13.3\n"
time="2023-07-18T13:54:44+01:00" level=debug msg="Running SSH command: timeout 30s oc get clusterversion version -o jsonpath=\"{['spec']['clusterID']}\" --context admin --cluster crc --kubeconfig /opt/kubeconfig"
time="2023-07-18T13:54:45+01:00" level=debug msg="SSH command results: err: <nil>, output: "
time="2023-07-18T13:54:45+01:00" level=info msg="Updating cluster ID..."
time="2023-07-18T13:54:45+01:00" level=debug msg="Running SSH command: timeout 30s oc patch clusterversion version -p '{\"spec\":{\"clusterID\":\"64b0bd8f-b831-4ffc-b279-850cbc8da193\"}}' --type merge --context admin --cluster crc --kubeconfig /opt/kubeconfig"
time="2023-07-18T13:54:45+01:00" level=debug msg="SSH command results: err: <nil>, output: clusterversion.config.openshift.io/version patched\n"
time="2023-07-18T13:54:45+01:00" level=debug msg="Creating /tmp/routes-controller.json with permissions 0644 in the CRC VM"
time="2023-07-18T13:54:45+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:45+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:45+01:00" level=debug msg="Running SSH command: timeout 30s oc apply -f /tmp/routes-controller.json --context admin --cluster crc --kubeconfig /opt/kubeconfig"
time="2023-07-18T13:54:46+01:00" level=debug msg="SSH command results: err: <nil>, output: pod/routes-controller created\n"
time="2023-07-18T13:54:46+01:00" level=debug msg="Waiting for availability of resource type 'configmaps'"
time="2023-07-18T13:54:46+01:00" level=debug msg="retry loop: attempt 0"
time="2023-07-18T13:54:46+01:00" level=debug msg="Running SSH command: timeout 5s oc get configmaps --context admin --cluster crc --kubeconfig /opt/kubeconfig"
time="2023-07-18T13:54:46+01:00" level=debug msg="SSH command results: err: <nil>, output: NAME                       DATA   AGE\nkube-root-ca.crt           1      28d\nopenshift-service-ca.crt   1      28d\n"
time="2023-07-18T13:54:46+01:00" level=debug msg="NAME                       DATA   AGE\nkube-root-ca.crt           1      28d\nopenshift-service-ca.crt   1      28d\n"
time="2023-07-18T13:54:46+01:00" level=debug msg="Running SSH command: timeout 30s oc get configmaps admin-kubeconfig-client-ca -n openshift-config -o jsonpath=\"{.data.ca-bundle\\.crt}\" --context admin --cluster crc --kubeconfig /opt/kubeconfig"
time="2023-07-18T13:54:46+01:00" level=debug msg="SSH command results: err: <nil>, output: -----BEGIN CERTIFICATE-----\nMIIDMDCCAhigAwIBAgIIRD1baiHDRZgwDQYJKoZIhvcNAQELBQAwNjESMBAGA1UE\nCxMJb3BlbnNoaWZ0MSAwHgYDVQQDExdhZG1pbi1rdWJlY29uZmlnLXNpZ25lcjAe\nFw0yMzA2MjAwNDU0MDBaFw0zMzA2MTcwNDU0MDBaMDYxEjAQBgNVBAsTCW9wZW5z\naGlmdDEgMB4GA1UEAxMXYWRtaW4ta3ViZWNvbmZpZy1zaWduZXIwggEiMA0GCSqG\nSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCv9/A8yBrfvm9GiFimRySiUBRYfMunOma8\n+xmMOYL7LWobENyXYWWgjIo5bJoaG2xoXCoKkeg38hp4FEZauZhwnuJEkqHmQznB\nNuvCRgXD/5PHW6IxIMAyjft3k2lxpsGDepw5v6i1pqnoctbHrZVDaQSl9lCjzWLv\nnhjzLCgfZyT+FR4/6jphlDFxi8/n62v7eKMAV2oNVE3gE7p1+kXaTODdPn25t0IK\nQPiOlXBBgjw/0Fue/K7rv28XZuNL0pol1gJTJwf5S6RzROJhKud9tn3OTojNb5Su\nMhpZlNM4FRUzrMGHFEnLpWsuNevB/Ey+08SH7zNfn1PyKeOMlAJBAgMBAAGjQjBA\nMA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBQ3u06K\nxUWFXvvV/qubIbUwtEtoOjANBgkqhkiG9w0BAQsFAAOCAQEAWTPuyO+oMLOAD2on\nx0P8MaqTxFt1+P/SXGgbuwKOfBaBBDfMd9hcI6dvWQw2sur6wsb+0btY9lP9qePq\n712rnXcfTFzFPkACbWp2oQSG/OHl2OzPZ0w/ec6+CZ45tkk8aB1QeUTUI+GeUmen\nUgU+kOJyOn4Ej/B8Rdk8ZZ/KUK7CxM0jodduqtF89urovBOuiQNdSKPDY0OmHu7R\nn2M7nKhdfUC3k6Z/Us+7TU1Uj9Mm1Mr8YnL+qrxdVPHeqy4YQarSHjICudfMV2IT\nIvlAvOUhLblvRxb+a3nFrT90YRdb7VF9W4/pGnbcHTDelG3J26Q51sO7I5LvjL7D\nzEZADg==\n-----END CERTIFICATE-----\n"
time="2023-07-18T13:54:46+01:00" level=info msg="Updating root CA cert to admin-kubeconfig-client-ca configmap..."
time="2023-07-18T13:54:46+01:00" level=debug msg="Running SSH command: timeout 30s oc patch configmap admin-kubeconfig-client-ca -n openshift-config --patch '{\"data\": {\"ca-bundle.crt\": \"-----BEGIN CERTIFICATE-----\\nMIIDSjCCAjKgAwIBAgIIZSGf5c7azCQwDQYJKoZIhvcNAQELBQAwPTESMBAGA1UE\\nCxMJb3BlbnNoaWZ0MScwJQYDVQQDEx5hZG1pbi1rdWJlY29uZmlnLXNpZ25lci1j\\ndXN0b20wHhcNMjMwNzE4MTI1NDQ2WhcNMzMwNzE1MTI1NDQ2WjA9MRIwEAYDVQQL\\nEwlvcGVuc2hpZnQxJzAlBgNVBAMTHmFkbWluLWt1YmVjb25maWctc2lnbmVyLWN1\\nc3RvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANzhRfkNjhWPO+/G\\nwcxRMAPZvJIMjB0YXrE1zAldPOJ9z5J9p5y337QiEgaJoRg70lJeR5bb63jZX0zw\\njLgpiH6SMUxBBvaGxjWVF5uFY4dJ59VJVs/8ZRQwR9orT3rf4ZOptVWs680ndy6w\\nxsNCHStibx7UlwSPzziwOBMYX0FXLKvdxn7TJX1NRwUTxaONmF7T6+SV2idFotet\\ne+wg+pFNp6FgwC9sXnKqGH2bE3i/atXFQDtq46yTkHhGppAsy4rMYz9z9i4hrITD\\nAvlGnfaTwHP+1DBWj7XmrnVbOYgfuiPM3SQ8F0rBwbXzd4PqjHCfd4n67z0txD8R\\nqRDBIF8CAwEAAaNOMEwwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB/wQFMAMBAf8w\\nKQYDVR0OBCIEIBkvGeYZRXqfAJWlVPnBFBOJdhlW4fVhQm26tABMtjqnMA0GCSqG\\nSIb3DQEBCwUAA4IBAQBGCXLUVNCAiRBgGmxOPqNgVP2N5TIjXiWM5LtEJQy+/vHz\\nHxzO/EwmAojHrcvaPmfnEd/f6rZYSpsj+LCp94AryLRDOS3bZ4P5dP1F2fzAI8cF\\nNEn8JKlvPWN5IX91Y16oBW4+rEJSWeIV716/Aa+fsmxGpoFnM8rAnwNEuUmxv03n\\n5XK9m0Hpm07MJVawfH0TiL1r6JTYKsTZDjdF/X0HuwaZnPJGvnbALJCIK2CECpKw\\nZFTe3/HXp3Li1wiZyc1ejMB7VLsoU7qc6rqWcQuGIhDpxwVJCJ4HwJZA3DVBZKgm\\nTAIenQ5mfqs5h4HrsyaxdyD/rvc1HlMUznkVc7Hc\\n-----END CERTIFICATE-----\\n\"}}' --context admin --cluster crc --kubeconfig /opt/kubeconfig"
time="2023-07-18T13:54:47+01:00" level=debug msg="SSH command results: err: <nil>, output: configmap/admin-kubeconfig-client-ca patched\n"
time="2023-07-18T13:54:47+01:00" level=debug msg="Creating /opt/kubeconfig with permissions 0644 in the CRC VM"
time="2023-07-18T13:54:47+01:00" level=debug msg="Running SSH command: <hidden>"
time="2023-07-18T13:54:47+01:00" level=debug msg="SSH command succeeded"
time="2023-07-18T13:54:47+01:00" level=info msg="Starting openshift instance... [waiting for the cluster to stabilize]"
time="2023-07-18T13:54:47+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:54:47+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:54:47+01:00" level=debug msg="dns operator not available, Reason: DNSUnavailable"
time="2023-07-18T13:54:47+01:00" level=debug msg="dns operator is still progressing, Reason: DNSReportsProgressingIsTrue"
time="2023-07-18T13:54:47+01:00" level=debug msg="dns operator is degraded, Reason: DNSDegraded"
time="2023-07-18T13:54:47+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:54:47+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:54:47+01:00" level=debug msg="ingress operator is still progressing, Reason: Reconciling"
time="2023-07-18T13:54:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:54:47+01:00" level=debug msg="kube-storage-version-migrator operator is still progressing, Reason: KubeStorageVersionMigrator_Deploying"
time="2023-07-18T13:54:47+01:00" level=debug msg="kube-storage-version-migrator operator not available, Reason: KubeStorageVersionMigrator_Deploying"
time="2023-07-18T13:54:47+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:54:47+01:00" level=debug msg="openshift-apiserver operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady"
time="2023-07-18T13:54:47+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:54:47+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:54:47+01:00" level=debug msg="operator-lifecycle-manager-packageserver operator not available, Reason: ClusterServiceVersionNotSucceeded"
time="2023-07-18T13:54:47+01:00" level=debug msg="operator-lifecycle-manager-packageserver operator is still progressing, Reason: "
time="2023-07-18T13:54:47+01:00" level=debug msg="service-ca operator is still progressing, Reason: _ManagedDeploymentsAvailable"
time="2023-07-18T13:54:47+01:00" level=info msg="8 operators are progressing: dns, image-registry, ingress, kube-storage-version-migrator, network, ..."
time="2023-07-18T13:55:17+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:55:17+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:55:17+01:00" level=debug msg="dns operator not available, Reason: DNSUnavailable"
time="2023-07-18T13:55:17+01:00" level=debug msg="dns operator is still progressing, Reason: DNSReportsProgressingIsTrue"
time="2023-07-18T13:55:17+01:00" level=debug msg="dns operator is degraded, Reason: DNSDegraded"
time="2023-07-18T13:55:17+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:55:17+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:55:17+01:00" level=debug msg="ingress operator is still progressing, Reason: Reconciling"
time="2023-07-18T13:55:17+01:00" level=debug msg="ingress operator is degraded, Reason: IngressDegraded"
time="2023-07-18T13:55:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:55:17+01:00" level=debug msg="kube-storage-version-migrator operator is still progressing, Reason: KubeStorageVersionMigrator_Deploying"
time="2023-07-18T13:55:17+01:00" level=debug msg="kube-storage-version-migrator operator not available, Reason: KubeStorageVersionMigrator_Deploying"
time="2023-07-18T13:55:17+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:55:17+01:00" level=debug msg="openshift-apiserver operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady"
time="2023-07-18T13:55:17+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:55:17+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:55:17+01:00" level=debug msg="operator-lifecycle-manager-packageserver operator not available, Reason: ClusterServiceVersionNotSucceeded"
time="2023-07-18T13:55:17+01:00" level=debug msg="operator-lifecycle-manager-packageserver operator is still progressing, Reason: "
time="2023-07-18T13:55:17+01:00" level=debug msg="service-ca operator is still progressing, Reason: _ManagedDeploymentsAvailable"
time="2023-07-18T13:55:17+01:00" level=info msg="8 operators are progressing: dns, image-registry, ingress, kube-storage-version-migrator, network, ..."
time="2023-07-18T13:55:47+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:55:47+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:55:47+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:55:47+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:55:47+01:00" level=debug msg="ingress operator is still progressing, Reason: Reconciling"
time="2023-07-18T13:55:47+01:00" level=debug msg="ingress operator is degraded, Reason: IngressDegraded"
time="2023-07-18T13:55:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:55:47+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:55:47+01:00" level=debug msg="openshift-apiserver operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady"
time="2023-07-18T13:55:47+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:55:47+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:55:47+01:00" level=debug msg="service-ca operator is still progressing, Reason: _ManagedDeploymentsAvailable"
time="2023-07-18T13:55:47+01:00" level=info msg="5 operators are progressing: image-registry, ingress, network, openshift-controller-manager, service-ca"
time="2023-07-18T13:56:17+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:56:17+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:56:17+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:56:17+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:56:17+01:00" level=debug msg="ingress operator is still progressing, Reason: Reconciling"
time="2023-07-18T13:56:17+01:00" level=debug msg="ingress operator is degraded, Reason: IngressDegraded"
time="2023-07-18T13:56:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:56:17+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:56:17+01:00" level=debug msg="openshift-apiserver operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady"
time="2023-07-18T13:56:17+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:56:17+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:56:17+01:00" level=debug msg="service-ca operator is still progressing, Reason: _ManagedDeploymentsAvailable"
time="2023-07-18T13:56:17+01:00" level=info msg="5 operators are progressing: image-registry, ingress, network, openshift-controller-manager, service-ca"
time="2023-07-18T13:56:47+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:56:47+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:56:47+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:56:47+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:56:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:56:47+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:56:47+01:00" level=debug msg="openshift-apiserver operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady"
time="2023-07-18T13:56:47+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:56:47+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:56:47+01:00" level=debug msg="service-ca operator is still progressing, Reason: _ManagedDeploymentsAvailable"
time="2023-07-18T13:56:47+01:00" level=info msg="4 operators are progressing: image-registry, network, openshift-controller-manager, service-ca"
time="2023-07-18T13:57:17+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:57:17+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:57:17+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:57:17+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:57:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:57:17+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:57:17+01:00" level=debug msg="openshift-apiserver operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady"
time="2023-07-18T13:57:17+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:57:17+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:57:17+01:00" level=debug msg="service-ca operator is still progressing, Reason: _ManagedDeploymentsAvailable"
time="2023-07-18T13:57:17+01:00" level=info msg="4 operators are progressing: image-registry, network, openshift-controller-manager, service-ca"
time="2023-07-18T13:57:47+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:57:47+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:57:47+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:57:47+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:57:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:57:47+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:57:47+01:00" level=debug msg="openshift-apiserver operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady"
time="2023-07-18T13:57:47+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:57:47+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:57:47+01:00" level=debug msg="service-ca operator is still progressing, Reason: _ManagedDeploymentsAvailable"
time="2023-07-18T13:57:47+01:00" level=info msg="4 operators are progressing: image-registry, network, openshift-controller-manager, service-ca"
time="2023-07-18T13:58:17+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:58:17+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:58:17+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:58:17+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:58:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:58:17+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:58:17+01:00" level=debug msg="openshift-apiserver operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady"
time="2023-07-18T13:58:17+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:58:17+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:58:17+01:00" level=debug msg="service-ca operator is still progressing, Reason: _ManagedDeploymentsAvailable"
time="2023-07-18T13:58:17+01:00" level=info msg="4 operators are progressing: image-registry, network, openshift-controller-manager, service-ca"
time="2023-07-18T13:58:47+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:58:47+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:58:47+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:58:47+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:58:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:58:47+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:58:47+01:00" level=debug msg="openshift-apiserver operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady"
time="2023-07-18T13:58:47+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:58:47+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:58:47+01:00" level=debug msg="service-ca operator is still progressing, Reason: _ManagedDeploymentsAvailable"
time="2023-07-18T13:58:47+01:00" level=info msg="4 operators are progressing: image-registry, network, openshift-controller-manager, service-ca"
time="2023-07-18T13:59:17+01:00" level=debug msg="authentication operator not available, Reason: APIServerDeployment_NoPod::APIServices_PreconditionNotReady::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable::OAuthServerServiceEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:59:17+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:59:17+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:59:17+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:59:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:59:17+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:59:17+01:00" level=debug msg="openshift-controller-manager operator is still progressing, Reason: _DesiredStateNotYetAchieved"
time="2023-07-18T13:59:17+01:00" level=debug msg="openshift-controller-manager operator not available, Reason: _NoPodsAvailable"
time="2023-07-18T13:59:17+01:00" level=info msg="3 operators are progressing: image-registry, network, openshift-controller-manager"
time="2023-07-18T13:59:47+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod::OAuthServerRouteEndpointAccessibleController_EndpointUnavailable"
time="2023-07-18T13:59:47+01:00" level=debug msg="console operator is degraded, Reason: OAuthClientSync_FailedGet::RouteHealth_RouteNotAdmitted::SyncLoopRefresh_FailedGet"
time="2023-07-18T13:59:47+01:00" level=debug msg="console operator not available, Reason: RouteHealth_RouteNotAdmitted"
time="2023-07-18T13:59:47+01:00" level=debug msg="image-registry operator not available, Reason: NoReplicasAvailable::NodeCADaemonNoAvailableReplicas"
time="2023-07-18T13:59:47+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted::NodeCADaemonUnavailable"
time="2023-07-18T13:59:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T13:59:47+01:00" level=debug msg="network operator is still progressing, Reason: Deploying"
time="2023-07-18T13:59:47+01:00" level=info msg="2 operators are progressing: image-registry, network"
time="2023-07-18T14:00:17+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod"
time="2023-07-18T14:00:17+01:00" level=debug msg="image-registry operator is still progressing, Reason: DeploymentNotCompleted"
time="2023-07-18T14:00:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T14:00:17+01:00" level=info msg="Operator image-registry is progressing"
time="2023-07-18T14:00:47+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod"
time="2023-07-18T14:00:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T14:00:47+01:00" level=info msg="Operator authentication is not yet available"
time="2023-07-18T14:01:17+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod"
time="2023-07-18T14:01:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T14:01:17+01:00" level=info msg="Operator authentication is not yet available"
time="2023-07-18T14:01:47+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod"
time="2023-07-18T14:01:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T14:01:47+01:00" level=info msg="Operator authentication is not yet available"
time="2023-07-18T14:02:17+01:00" level=debug msg="authentication operator is degraded, Reason: OAuthServerDeployment_UnavailablePod"
time="2023-07-18T14:02:17+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod"
time="2023-07-18T14:02:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T14:02:17+01:00" level=info msg="Operator authentication is degraded"
time="2023-07-18T14:02:47+01:00" level=debug msg="authentication operator is degraded, Reason: OAuthServerDeployment_UnavailablePod"
time="2023-07-18T14:02:47+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod"
time="2023-07-18T14:02:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T14:02:47+01:00" level=info msg="Operator authentication is degraded"
time="2023-07-18T14:03:17+01:00" level=debug msg="authentication operator is degraded, Reason: OAuthServerDeployment_UnavailablePod"
time="2023-07-18T14:03:17+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod"
time="2023-07-18T14:03:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T14:03:17+01:00" level=info msg="Operator authentication is degraded"
time="2023-07-18T14:03:47+01:00" level=debug msg="authentication operator is degraded, Reason: OAuthServerDeployment_UnavailablePod"
time="2023-07-18T14:03:47+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod"
time="2023-07-18T14:03:47+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T14:03:47+01:00" level=info msg="Operator authentication is degraded"
time="2023-07-18T14:04:17+01:00" level=debug msg="authentication operator is degraded, Reason: OAuthServerDeployment_UnavailablePod"
time="2023-07-18T14:04:17+01:00" level=debug msg="authentication operator not available, Reason: OAuthServerDeployment_NoPod"
time="2023-07-18T14:04:17+01:00" level=debug msg="Unexpected operator status for ingress: EvaluationConditionsDetected"
time="2023-07-18T14:04:17+01:00" level=info msg="Operator authentication is degraded"
time="2023-07-18T14:04:47+01:00" level=error msg="Cluster is not ready: cluster operators are still not stable after 10m0.5790021s"
time="2023-07-18T14:04:47+01:00" level=info msg="Adding crc-admin and crc-developer contexts to kubeconfig..."
time="2023-07-18T14:04:48+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\" -Name ReleaseId).ReleaseId'"
time="2023-07-18T14:04:48+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\").EditionID'"
time="2023-07-18T14:32:07+01:00" level=debug msg="CRC version: 2.23.0+ddcfe8\n"
time="2023-07-18T14:32:07+01:00" level=debug msg="OpenShift version: 4.13.3\n"
time="2023-07-18T14:32:07+01:00" level=debug msg="Podman version: 4.4.4\n"
time="2023-07-18T14:32:07+01:00" level=debug msg="Running 'crc oc-env'"
time="2023-07-18T14:32:07+01:00" level=debug msg="Checking file: C:\\Users\\skumar2382\\.crc\\machines\\crc\\.crc-exist"
time="2023-07-18T14:32:07+01:00" level=debug msg="Checking file: C:\\Users\\skumar2382\\.crc\\machines\\crc\\.crc-exist"
time="2023-07-18T14:32:07+01:00" level=debug msg="Running 'Hyper-V\\Get-VM crc | Select-Object -ExpandProperty State'"
time="2023-07-18T14:32:08+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\" -Name ReleaseId).ReleaseId'"
time="2023-07-18T14:32:08+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\").EditionID'"
time="2023-07-18T14:32:20+01:00" level=debug msg="CRC version: 2.23.0+ddcfe8\n"
time="2023-07-18T14:32:20+01:00" level=debug msg="OpenShift version: 4.13.3\n"
time="2023-07-18T14:32:20+01:00" level=debug msg="Podman version: 4.4.4\n"
time="2023-07-18T14:32:20+01:00" level=debug msg="Running 'crc oc-env'"
time="2023-07-18T14:32:20+01:00" level=debug msg="Checking file: C:\\Users\\skumar2382\\.crc\\machines\\crc\\.crc-exist"
time="2023-07-18T14:32:20+01:00" level=debug msg="Checking file: C:\\Users\\skumar2382\\.crc\\machines\\crc\\.crc-exist"
time="2023-07-18T14:32:20+01:00" level=debug msg="Running 'Hyper-V\\Get-VM crc | Select-Object -ExpandProperty State'"
time="2023-07-18T14:32:22+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\" -Name ReleaseId).ReleaseId'"
time="2023-07-18T14:32:22+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\").EditionID'"
time="2023-07-18T16:23:02+01:00" level=debug msg="CRC version: 2.23.0+ddcfe8\n"
time="2023-07-18T16:23:02+01:00" level=debug msg="OpenShift version: 4.13.3\n"
time="2023-07-18T16:23:02+01:00" level=debug msg="Podman version: 4.4.4\n"
time="2023-07-18T16:23:02+01:00" level=debug msg="Running 'crc version'"
time="2023-07-18T16:23:03+01:00" level=debug msg="No new version available. The latest version is 2.23.0"
time="2023-07-18T16:23:03+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\" -Name ReleaseId).ReleaseId'"
time="2023-07-18T16:23:03+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\").EditionID'"
time="2023-07-18T16:23:29+01:00" level=debug msg="CRC version: 2.23.0+ddcfe8"
time="2023-07-18T16:23:29+01:00" level=debug msg="OpenShift version: 4.13.3"
time="2023-07-18T16:23:29+01:00" level=debug msg="Podman version: 4.4.4"
time="2023-07-18T16:23:29+01:00" level=debug msg="Running 'crc status'"
time="2023-07-18T16:23:32+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\" -Name ReleaseId).ReleaseId'"
time="2023-07-18T16:23:32+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\").EditionID'"
time="2023-07-18T16:24:08+01:00" level=debug msg="CRC version: 2.23.0+ddcfe8\n"
time="2023-07-18T16:24:08+01:00" level=debug msg="OpenShift version: 4.13.3\n"
time="2023-07-18T16:24:08+01:00" level=debug msg="Podman version: 4.4.4\n"
time="2023-07-18T16:24:08+01:00" level=debug msg="Running 'crc config view'"
time="2023-07-18T16:24:08+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\" -Name ReleaseId).ReleaseId'"
time="2023-07-18T16:24:08+01:00" level=debug msg="Running '(Get-ItemProperty -Path \"HKLM:\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion

Before gather the logs try following if that fix your issue
```bash
$ crc delete -f
$ crc cleanup
$ crc setup
$ crc start --log-level debug

Please consider posting the output of crc start --log-level debug on http://gist.github.com/ and post the link in the issue.

praveenkumar commented 1 year ago

@santoshnt1 I can see the status of cluster is running which means all the cluster operators are in Ready state. what is the output of oc get pods -n openshift-marketplace ?

santoshnt1 commented 1 year ago

Hi Praveen, please find the output below

PS C:\Users\skumar2382> oc get pods -n openshift-marketplace
NAME                                    READY   STATUS             RESTARTS   AGE
certified-operators-7mjj2               0/1     ImagePullBackOff   3          27d
certified-operators-sgd2m               0/1     ImagePullBackOff   0          26d
community-operators-8klmh               0/1     ImagePullBackOff   0          26d
community-operators-rjft7               0/1     ImagePullBackOff   3          27d
marketplace-operator-76cc47dfd5-gsb2t   1/1     Running            2          27d
redhat-marketplace-5qft2                0/1     ImagePullBackOff   0          26d
redhat-marketplace-z6kxv                0/1     ImagePullBackOff   3          27d
redhat-operators-r6zjp                  0/1     ImagePullBackOff   3          27d
redhat-operators-tbq7w                  0/1     ImagePullBackOff   0          26d

Thanks Santosh

cfergeau commented 1 year ago

Do you need to use a http/https proxy for external network access?

santoshnt1 commented 1 year ago

no, no proxy required

santoshnt1 commented 1 year ago

Adding pod description it may gives you more info about issue 

PS C:\Users\skumar2382> oc describe pod redhat-operators-r6zjp -n openshift-marketplace
Name:                 redhat-operators-r6zjp
Namespace:            openshift-marketplace
Priority:             2000000000
Priority Class Name:  system-cluster-critical
Service Account:      redhat-operators
Node:                 crc-ccssl-master-0/192.168.126.11
Start Time:           Tue, 20 Jun 2023 06:05:01 +0100
Labels:               olm.catalogSource=redhat-operators
                      olm.pod-spec-hash=7dd4b897fb
Annotations:          cluster-autoscaler.kubernetes.io/safe-to-evict: true
                      k8s.v1.cni.cncf.io/network-status:
                        [{
                            "name": "openshift-sdn",
                            "interface": "eth0",
                            "ips": [
                                "10.217.0.51"
                            ],
                            "default": true,
                            "dns": {}
                        }]
                      openshift.io/scc: restricted-v2
                      operatorframework.io/managed-by: marketplace-operator
                      seccomp.security.alpha.kubernetes.io/pod: runtime/default
Status:               Running
IP:                   10.217.0.51
IPs:
  IP:  10.217.0.51
Containers:
  registry-server:
    Container ID:
    Image:          registry.redhat.io/redhat/redhat-operator-index:v4.13
    Image ID:
    Port:           50051/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ImagePullBackOff
    Last State:     Terminated
      Reason:       ContainerStatusUnknown
      Message:      The container could not be located when the pod was deleted.  The container used to be Running
      Exit Code:    137
      Started:      Mon, 01 Jan 0001 00:00:00 +0000
      Finished:     Mon, 01 Jan 0001 00:00:00 +0000
    Ready:          False
    Restart Count:  3
    Requests:
      cpu:        10m
      memory:     50Mi
    Liveness:     exec [grpc_health_probe -addr=:50051] delay=10s timeout=5s period=10s #success=1 #failure=3
    Readiness:    exec [grpc_health_probe -addr=:50051] delay=5s timeout=5s period=10s #success=1 #failure=3
    Startup:      exec [grpc_health_probe -addr=:50051] delay=0s timeout=5s period=10s #success=1 #failure=10
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wzjk5 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kube-api-access-wzjk5:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
    ConfigMapName:           openshift-service-ca.crt
    ConfigMapOptional:       <nil>
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
                             node-role.kubernetes.io/master=
Tolerations:                 node-role.kubernetes.io/master:NoSchedule op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 120s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 120s
Events:
  Type     Reason          Age                   From               Message
  ----     ------          ----                  ----               -------
  Normal   Scheduled       28d                   default-scheduler  Successfully assigned openshift-marketplace/redhat-operators-r6zjp to crc-ccssl-master-0
  Normal   AddedInterface  28d                   multus             Add eth0 [10.217.0.40/23] from openshift-sdn
  Normal   Pulling         28d                   kubelet            Pulling image "registry.redhat.io/redhat/redhat-operator-index:v4.13"
  Normal   Pulled          28d                   kubelet            Successfully pulled image "registry.redhat.io/redhat/redhat-operator-index:v4.13" in 28.092328789s (28.092343535s including waiting)
  Normal   Created         28d                   kubelet            Created container registry-server
  Normal   Started         28d                   kubelet            Started container registry-server
  Warning  NodeNotReady    27d                   node-controller    Node is not ready
  Normal   AddedInterface  27d                   multus             Add eth0 [10.217.0.17/23] from openshift-sdn
  Normal   Pulling         27d                   kubelet            Pulling image "registry.redhat.io/redhat/redhat-operator-index:v4.13"
  Normal   Pulled          27d                   kubelet            Successfully pulled image "registry.redhat.io/redhat/redhat-operator-index:v4.13" in 1.010731458s (1.010751857s including waiting)
  Normal   Created         27d                   kubelet            Created container registry-server
  Normal   Started         27d                   kubelet            Started container registry-server
  Normal   AddedInterface  27d                   multus             Add eth0 [10.217.0.10/23] from openshift-sdn
  Normal   Pulling         27d                   kubelet            Pulling image "registry.redhat.io/redhat/redhat-operator-index:v4.13"
  Normal   Pulled          27d                   kubelet            Successfully pulled image "registry.redhat.io/redhat/redhat-operator-index:v4.13" in 907.556869ms (907.566931ms including waiting)
  Normal   Created         27d                   kubelet            Created container registry-server
  Normal   Started         27d                   kubelet            Started container registry-server
  Warning  NodeNotReady    27d                   node-controller    Node is not ready
  Normal   AddedInterface  42m                   multus             Add eth0 [10.217.0.32/23] from openshift-sdn
  Warning  Failed          40m (x6 over 42m)     kubelet            Error: ImagePullBackOff
  Warning  Failed          40m (x4 over 42m)     kubelet            Error: ErrImagePull
  Warning  Failed          40m (x4 over 42m)     kubelet            Failed to pull image "registry.redhat.io/redhat/redhat-operator-index:v4.13": rpc error: code = Unknown desc = pinging container registry registry.redhat.io: Get "https://registry.redhat.io/v2/": x509: certificate signed by unknown authority
  Normal   Pulling         40m (x4 over 42m)     kubelet            Pulling image "registry.redhat.io/redhat/redhat-operator-index:v4.13"
  Normal   BackOff         31m (x47 over 42m)    kubelet            Back-off pulling image "registry.redhat.io/redhat/redhat-operator-index:v4.13"
  Normal   AddedInterface  25m                   multus             Add eth0 [10.217.0.51/23] from openshift-sdn
  Warning  Failed          23m (x4 over 25m)     kubelet            Failed to pull image "registry.redhat.io/redhat/redhat-operator-index:v4.13": rpc error: code = Unknown desc = pinging container registry registry.redhat.io: Get "https://registry.redhat.io/v2/": x509: certificate signed by unknown authority
  Warning  Failed          23m (x4 over 25m)     kubelet            Error: ErrImagePull
  Warning  Failed          23m (x6 over 25m)     kubelet            Error: ImagePullBackOff
  Normal   Pulling         9m24s (x8 over 25m)   kubelet            Pulling image "registry.redhat.io/redhat/redhat-operator-index:v4.13"
  Normal   BackOff         4m31s (x90 over 25m)  kubelet            Back-off pulling image "registry.redhat.io/redhat/redhat-operator-index:v4.13"
cfergeau commented 1 year ago 
Get "https://registry.redhat.io/v2/": x509: certificate signed by unknown authority
  Warning  Failed          23m (x4 over 25m)     kubelet            Error: ErrImagePull

is definitely problematic, but I don't know why this would be happening.

praveenkumar commented 1 year ago

@santoshnt1 can you run openssl s_client -connect registry.redhat.io:443 on your host and also in the VM ( you can access the VM using https://github.com/crc-org/crc/wiki/Debugging-guide ) because this seems to like a proxy in place without your knowledge.

santoshnt1 commented 1 year ago

HI Praveen, please find the output below : From Host

ED(00000003)
depth=2 C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalerthree.net), emailAddress = support@zscaler.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=1 C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
verify return:1
depth=0 C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", CN = registry.redhat.io
verify return:1
---
Certificate chain
 0 s:C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", CN = registry.redhat.io
   i:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jul 15 09:25:06 2023 GMT; NotAfter: Jul 29 09:25:06 2023 GMT
 1 s:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
   i:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalerthree.net), emailAddress = support@zscaler.com
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jul 15 09:25:06 2023 GMT; NotAfter: Jul 29 09:25:06 2023 GMT
 2 s:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalerthree.net), emailAddress = support@zscaler.com
   i:C = US, ST = California, L = San Jose, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Root CA, emailAddress = support@zscaler.com
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jun  5 05:32:44 2020 GMT; NotAfter: Jun 23 05:32:44 2041 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIE0jCCA7qgAwIBAgIQbb7//wHIvH+E0erm7PFd5zANBgkqhkiG9w0BAQsFADCB
jzELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFTATBgNVBAoMDFpz
Y2FsZXIgSW5jLjEVMBMGA1UECwwMWnNjYWxlciBJbmMuMT0wOwYDVQQDDDRac2Nh
bGVyIEludGVybWVkaWF0ZSBSb290IENBICh6c2NhbGVydGhyZWUubmV0KSAodCkg
MB4XDTIzMDcxNTA5MjUwNloXDTIzMDcyOTA5MjUwNlowbTELMAkGA1UEBhMCVVMx
FzAVBgNVB-bash: [core@crc-ccssl-master-0: command not found
AgTDk[user@GB-5CG30917V4 skumar2382]$ CONNECTED(00000003)
-bash: syntax error near unexpected token `00000003'
[user@GB-5CG30917V4 skumar2382]$ depth=2 C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalerthree.net), emailAddress = support@zscaler.com
-bash: syntax error near unexpected token `('
[user@GB-5CG30917V4 skumar2382]$ verify error:num=20:unable to get local issuer certificate
ENhcm9saW5hMRAwDgYDVQQHEwdSYWxlaWdoMRYwFAYD
VQQKEw1SZWQgSGF0LCBJbmMuMRswGQYDVQQDExJyZWdpc3RyeS5yZWRoYXQuaW8w
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC6tc6RnE3qF7dWVFhgB/1Z
Az2CBYpF1ZF25Ju82TCqh/2A+JtTBmDVs+6y4wBCLqTS/bUK1OJlPrMadJoBc/yl
3L6O1dpit3hZmLOH3LMmmozF5NCsnRJrbPFuaIpKd473fe8Z0fCyO3JVTiLsJpol
Xe2+YMYiF3Hol09DVYJQ/pfk1jr1mMYTLa2hr2l6qwHk7O7Wcq1QUgAZxoneRcrB
S4IADRLO96UM7nM6rfF/dtBL8MWFVyK2BkyvZGALvXwk5P/MO3XGoU1rfrFxuXwu
sWBPidBnD4n+mJLVyC0e6hZn80edogOdHEjhGA9R08UKrnlv37Z+ri2poQFS1L9j
AgMBAAGjggFJMIIBRTCBvwYDVR0RBIG3MIG0ghJyZWdpc3RyeS5yZWRoYXQuaW+C
E3JlZ2lzdHJ5Ni5yZWRoYXQuaW+CHHJlZ2lzdHJ5Ni5jb25uZWN0LnJlZGhhdC5j
b22CG3JlZ2lzdHJ5Ni5hY2Nlc3MucmVkaGF0LmNvbYIbcmVnaXN0cnkuY29ubmVj
dC5yZWRoYXQuY29tghpyZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbYIVYWNjZXNz
LmNkbi5yZWRoYXQuY29tMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEF
BQcDAQYIKwYBBQUHAwIwCQYDVR0TBAIwADBHBgNVHR8EQDA+MDygOqA4hjZodHRw
Oi8vZ2F0ZXdheS56c2NhbGVydGhyZWUubmV0L3pzY2FsZXItenNjcmwtLTQtMS5j
cmwwDQYJKoZIhvcNAQELBQADggEBAHjHYL/F4WwkCWTzDgbvV8egx4WSauEU3gyF
pz++BXDsKYhGTWYz9qk/VowLFmpo9dc6lI9WvT6RmdOvinqRmZSif4rLs/016YEc
NevjDqTcKMExBJ0KUMYSzAqxPvdCqMLBeT1hzvtIYXGDTB34wKxUiUoRZhb/dit/
Us3fWpOFK/KC3kV9Fhgo2WyDhe1zTeozE5rZo+gUckEdzLbwwstlKhD2eGD0/a2a
Fa/n4rpnRFx6PNC0bekTfK72RtHLB/+VSRejciXCqZ6d0GfxYt//lQFFxSDvYiIN
LLxl9-bash: verify: command not found
EUh4jn[user@GB-5CG30917V4 skumar2382]$ verify return:1
G1PXPalBrsqjS2pFYvwSkk54KbkWpFsv838h9XFk=
-----END CERTIFICATE-----
subject=C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", CN = registry.redhat.io
issuer=C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 4133 bytes and written 757 bytes
Verification error: unable to get local issuer certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 20 (unable to get local issuer certificate)
----bash: verify: command not found
[user@GB-5CG30917V4 skumar2382]$ depth=1 C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
-bash: C: command not found
[user@GB-5CG30917V4 skumar2382]$ verify return:1
-bash: verify: command not found
[user@GB-5CG30917V4 skumar2382]$ depth=0 C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", CN = registry.redhat.io
-bash: C: command not found
[user@GB-5CG30917V4 skumar2382]$ verify return:1
-bash: verify: command not found
[user@GB-5CG30917V4 skumar2382]$ ---
-bash: ---: command not found
[user@GB-5CG30917V4 skumar2382]$ Certificate chain
-bash: Certificate: command not found
[user@GB-5CG30917V4 skumar2382]$  0 s:C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", CN = registry.redhat.io
-bash: 0: command not found
[user@GB-5CG30917V4 skumar2382]$    i:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
-bash: i:C: command not found
[user@GB-5CG30917V4 skumar2382]$    a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
-bash: syntax error near unexpected token `('
[user@GB-5CG30917V4 skumar2382]$    v:NotBefore: Jul 15 09:25:06 2023 GMT; NotAfter: Jul 29 09:25:06 2023 GMT
-bash: v:NotBefore:: command not found
-bash: NotAfter:: command not found
[user@GB-5CG30917V4 skumar2382]$  1 s:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
-bash: 1: command not found
[user@GB-5CG30917V4 skumar2382]$    i:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalerthree.net), emailAddress = support@zscaler.com
-bash: syntax error near unexpected token `('
[user@GB-5CG30917V4 skumar2382]$    a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
-bash: syntax error near unexpected token `('
[user@GB-5CG30917V4 skumar2382]$    v:NotBefore: Jul 15 09:25:06 2023 GMT; NotAfter: Jul 29 09:25:06 2023 GMT
-bash: v:NotBefore:: command not found
-bash: NotAfter:: command not found
[user@GB-5CG30917V4 skumar2382]$  2 s:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalerthree.net), emailAddress = support@zscaler.com
-bash: syntax error near unexpected token `('
[user@GB-5CG30917V4 skumar2382]$    i:C = US, ST = California, L = San Jose, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Root CA, emailAddress = support@zscaler.com
-bash: i:C: command not found
[user@GB-5CG30917V4 skumar2382]$    a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
-bash: syntax error near unexpected token `('
[user@GB-5CG30917V4 skumar2382]$    v:NotBefore: Jun  5 05:32:44 2020 GMT; NotAfter: Jun 23 05:32:44 2041 GMT
-bash: v:NotBefore:: command not found
-bash: NotAfter:: command not found
[user@GB-5CG30917V4 skumar2382]$ ---
-bash: ---: command not found
[user@GB-5CG30917V4 skumar2382]$ Server certificate
-bash: Server: command not found
[user@GB-5CG30917V4 skumar2382]$ -----BEGIN CERTIFICATE-----
-bash: -----BEGIN: command not found
[user@GB-5CG30917V4 skumar2382]$ MIIE0jCCA7qgAwIBAgIQbb7//wHIvH+E0erm7PFd5zANBgkqhkiG9w0BAQsFADCB
-bash: MIIE0jCCA7qgAwIBAgIQbb7//wHIvH+E0erm7PFd5zANBgkqhkiG9w0BAQsFADCB: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ jzELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFTATBgNVBAoMDFpz
-bash: jzELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFTATBgNVBAoMDFpz: command not found
[user@GB-5CG30917V4 skumar2382]$ Y2FsZXIgSW5jLjEVMBMGA1UECwwMWnNjYWxlciBJbmMuMT0wOwYDVQQDDDRac2Nh
-bash: Y2FsZXIgSW5jLjEVMBMGA1UECwwMWnNjYWxlciBJbmMuMT0wOwYDVQQDDDRac2Nh: command not found
[user@GB-5CG30917V4 skumar2382]$ bGVyIEludGVybWVkaWF0ZSBSb290IENBICh6c2NhbGVydGhyZWUubmV0KSAodCkg
-bash: bGVyIEludGVybWVkaWF0ZSBSb290IENBICh6c2NhbGVydGhyZWUubmV0KSAodCkg: command not found
[user@GB-5CG30917V4 skumar2382]$ MB4XDTIzMDcxNTA5MjUwNloXDTIzMDcyOTA5MjUwNlowbTELMAkGA1UEBhMCVVMx
-bash: MB4XDTIzMDcxNTA5MjUwNloXDTIzMDcyOTA5MjUwNlowbTELMAkGA1UEBhMCVVMx: command not found
[user@GB-5CG30917V4 skumar2382]$ FzAVBgNVBAgTDk5vcnRoIENhcm9saW5hMRAwDgYDVQQHEwdSYWxlaWdoMRYwFAYD
-bash: FzAVBgNVBAgTDk5vcnRoIENhcm9saW5hMRAwDgYDVQQHEwdSYWxlaWdoMRYwFAYD: command not found
[user@GB-5CG30917V4 skumar2382]$ VQQKEw1SZWQgSGF0LCBJbmMuMRswGQYDVQQDExJyZWdpc3RyeS5yZWRoYXQuaW8w
-bash: VQQKEw1SZWQgSGF0LCBJbmMuMRswGQYDVQQDExJyZWdpc3RyeS5yZWRoYXQuaW8w: command not found
[user@GB-5CG30917V4 skumar2382]$ ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC6tc6RnE3qF7dWVFhgB/1Z
-bash: ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC6tc6RnE3qF7dWVFhgB/1Z: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ Az2CBYpF1ZF25Ju82TCqh/2A+JtTBmDVs+6y4wBCLqTS/bUK1OJlPrMadJoBc/yl
-bash: Az2CBYpF1ZF25Ju82TCqh/2A+JtTBmDVs+6y4wBCLqTS/bUK1OJlPrMadJoBc/yl: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ 3L6O1dpit3hZmLOH3LMmmozF5NCsnRJrbPFuaIpKd473fe8Z0fCyO3JVTiLsJpol
-bash: 3L6O1dpit3hZmLOH3LMmmozF5NCsnRJrbPFuaIpKd473fe8Z0fCyO3JVTiLsJpol: command not found
[user@GB-5CG30917V4 skumar2382]$ Xe2+YMYiF3Hol09DVYJQ/pfk1jr1mMYTLa2hr2l6qwHk7O7Wcq1QUgAZxoneRcrB
-bash: Xe2+YMYiF3Hol09DVYJQ/pfk1jr1mMYTLa2hr2l6qwHk7O7Wcq1QUgAZxoneRcrB: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ S4IADRLO96UM7nM6rfF/dtBL8MWFVyK2BkyvZGALvXwk5P/MO3XGoU1rfrFxuXwu
-bash: S4IADRLO96UM7nM6rfF/dtBL8MWFVyK2BkyvZGALvXwk5P/MO3XGoU1rfrFxuXwu: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ sWBPidBnD4n+mJLVyC0e6hZn80edogOdHEjhGA9R08UKrnlv37Z+ri2poQFS1L9j
-bash: sWBPidBnD4n+mJLVyC0e6hZn80edogOdHEjhGA9R08UKrnlv37Z+ri2poQFS1L9j: command not found
[user@GB-5CG30917V4 skumar2382]$ AgMBAAGjggFJMIIBRTCBvwYDVR0RBIG3MIG0ghJyZWdpc3RyeS5yZWRoYXQuaW+C
-bash: AgMBAAGjggFJMIIBRTCBvwYDVR0RBIG3MIG0ghJyZWdpc3RyeS5yZWRoYXQuaW+C: command not found
[user@GB-5CG30917V4 skumar2382]$ E3JlZ2lzdHJ5Ni5yZWRoYXQuaW+CHHJlZ2lzdHJ5Ni5jb25uZWN0LnJlZGhhdC5j
-bash: E3JlZ2lzdHJ5Ni5yZWRoYXQuaW+CHHJlZ2lzdHJ5Ni5jb25uZWN0LnJlZGhhdC5j: command not found
[user@GB-5CG30917V4 skumar2382]$ b22CG3JlZ2lzdHJ5Ni5hY2Nlc3MucmVkaGF0LmNvbYIbcmVnaXN0cnkuY29ubmVj
-bash: b22CG3JlZ2lzdHJ5Ni5hY2Nlc3MucmVkaGF0LmNvbYIbcmVnaXN0cnkuY29ubmVj: command not found
[user@GB-5CG30917V4 skumar2382]$ dC5yZWRoYXQuY29tghpyZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbYIVYWNjZXNz
-bash: dC5yZWRoYXQuY29tghpyZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbYIVYWNjZXNz: command not found
[user@GB-5CG30917V4 skumar2382]$ LmNkbi5yZWRoYXQuY29tMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEF
-bash: LmNkbi5yZWRoYXQuY29tMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEF: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ BQcDAQYIKwYBBQUHAwIwCQYDVR0TBAIwADBHBgNVHR8EQDA+MDygOqA4hjZodHRw
-bash: BQcDAQYIKwYBBQUHAwIwCQYDVR0TBAIwADBHBgNVHR8EQDA+MDygOqA4hjZodHRw: command not found
[user@GB-5CG30917V4 skumar2382]$ Oi8vZ2F0ZXdheS56c2NhbGVydGhyZWUubmV0L3pzY2FsZXItenNjcmwtLTQtMS5j
-bash: Oi8vZ2F0ZXdheS56c2NhbGVydGhyZWUubmV0L3pzY2FsZXItenNjcmwtLTQtMS5j: command not found
[user@GB-5CG30917V4 skumar2382]$ cmwwDQYJKoZIhvcNAQELBQADggEBAHjHYL/F4WwkCWTzDgbvV8egx4WSauEU3gyF
-bash: cmwwDQYJKoZIhvcNAQELBQADggEBAHjHYL/F4WwkCWTzDgbvV8egx4WSauEU3gyF: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ pz++BXDsKYhGTWYz9qk/VowLFmpo9dc6lI9WvT6RmdOvinqRmZSif4rLs/016YEc
-bash: pz++BXDsKYhGTWYz9qk/VowLFmpo9dc6lI9WvT6RmdOvinqRmZSif4rLs/016YEc: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ NevjDqTcKMExBJ0KUMYSzAqxPvdCqMLBeT1hzvtIYXGDTB34wKxUiUoRZhb/dit/
-bash: NevjDqTcKMExBJ0KUMYSzAqxPvdCqMLBeT1hzvtIYXGDTB34wKxUiUoRZhb/dit/: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ Us3fWpOFK/KC3kV9Fhgo2WyDhe1zTeozE5rZo+gUckEdzLbwwstlKhD2eGD0/a2a
-bash: Us3fWpOFK/KC3kV9Fhgo2WyDhe1zTeozE5rZo+gUckEdzLbwwstlKhD2eGD0/a2a: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ Fa/n4rpnRFx6PNC0bekTfK72RtHLB/+VSRejciXCqZ6d0GfxYt//lQFFxSDvYiIN
-bash: Fa/n4rpnRFx6PNC0bekTfK72RtHLB/+VSRejciXCqZ6d0GfxYt//lQFFxSDvYiIN: No such file or directory
[user@GB-5CG30917V4 skumar2382]$ LLxl9EUh4jnG1PXPalBrsqjS2pFYvwSkk54KbkWpFsv838h9XFk=
[user@GB-5CG30917V4 skumar2382]$ -----END CERTIFICATE-----
-bash: -----END: command not found
[user@GB-5CG30917V4 skumar2382]$ subject=C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", CN = registry.redhat.io
-bash: =: command not found
[user@GB-5CG30917V4 skumar2382]$ issuer=C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
-bash: =: command not found
[user@GB-5CG30917V4 skumar2382]$ ---
-bash: ---: command not found
[user@GB-5CG30917V4 skumar2382]$ No client certificate CA names sent
-bash: No: command not found
[user@GB-5CG30917V4 skumar2382]$ Peer signing digest: SHA256
-bash: Peer: command not found
[user@GB-5CG30917V4 skumar2382]$ Peer signature type: RSA-PSS
-bash: Peer: command not found
[user@GB-5CG30917V4 skumar2382]$ Server Temp Key: ECDH, prime256v1, 256 bits
-bash: Server: command not found
[user@GB-5CG30917V4 skumar2382]$ ---
-bash: ---: command not found
[user@GB-5CG30917V4 skumar2382]$ SSL handshake has read 4133 bytes and written 757 bytes
-bash: SSL: command not found
[user@GB-5CG30917V4 skumar2382]$ Verification error: unable to get local issuer certificate
-bash: Verification: command not found
[user@GB-5CG30917V4 skumar2382]$ ---
-bash: ---: command not found
[user@GB-5CG30917V4 skumar2382]$ New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
-bash: New,: command not found
[user@GB-5CG30917V4 skumar2382]$ Server public key is 2048 bit
-bash: Server: command not found
[user@GB-5CG30917V4 skumar2382]$ Secure Renegotiation IS NOT supported
-bash: Secure: command not found
[user@GB-5CG30917V4 skumar2382]$ Compression: NONE
-bash: Compression:: command not found
[user@GB-5CG30917V4 skumar2382]$ Expansion: NONE
-bash: Expansion:: command not found
[user@GB-5CG30917V4 skumar2382]$ No ALPN negotiated
-bash: No: command not found
[user@GB-5CG30917V4 skumar2382]$ Early data was not sent
-bash: Early: command not found
[user@GB-5CG30917V4 skumar2382]$ Verify return code: 20 (unable to get local issuer certificate)
-bash: syntax error near unexpected token `('

from VM

[core@crc-ccssl-master-0 ~]$ openssl s_client -connect registry.redhat.io:443
CONNECTED(00000003)
depth=2 C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalerthree.net), emailAddress = support@zscaler.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=1 C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
verify return:1
depth=0 C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", CN = registry.redhat.io
verify return:1
---
Certificate chain
 0 s:C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", CN = registry.redhat.io
   i:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jul 15 09:25:06 2023 GMT; NotAfter: Jul 29 09:25:06 2023 GMT
 1 s:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
   i:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalerthree.net), emailAddress = support@zscaler.com
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jul 15 09:25:06 2023 GMT; NotAfter: Jul 29 09:25:06 2023 GMT
 2 s:C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalerthree.net), emailAddress = support@zscaler.com
   i:C = US, ST = California, L = San Jose, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Root CA, emailAddress = support@zscaler.com
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jun  5 05:32:44 2020 GMT; NotAfter: Jun 23 05:32:44 2041 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIE0jCCA7qgAwIBAgIQbb7//wHIvH+E0erm7PFd5zANBgkqhkiG9w0BAQsFADCB
jzELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFTATBgNVBAoMDFpz
Y2FsZXIgSW5jLjEVMBMGA1UECwwMWnNjYWxlciBJbmMuMT0wOwYDVQQDDDRac2Nh
bGVyIEludGVybWVkaWF0ZSBSb290IENBICh6c2NhbGVydGhyZWUubmV0KSAodCkg
MB4XDTIzMDcxNTA5MjUwNloXDTIzMDcyOTA5MjUwNlowbTELMAkGA1UEBhMCVVMx
FzAVBgNVBAgTDk5vcnRoIENhcm9saW5hMRAwDgYDVQQHEwdSYWxlaWdoMRYwFAYD
VQQKEw1SZWQgSGF0LCBJbmMuMRswGQYDVQQDExJyZWdpc3RyeS5yZWRoYXQuaW8w
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC6tc6RnE3qF7dWVFhgB/1Z
Az2CBYpF1ZF25Ju82TCqh/2A+JtTBmDVs+6y4wBCLqTS/bUK1OJlPrMadJoBc/yl
3L6O1dpit3hZmLOH3LMmmozF5NCsnRJrbPFuaIpKd473fe8Z0fCyO3JVTiLsJpol
Xe2+YMYiF3Hol09DVYJQ/pfk1jr1mMYTLa2hr2l6qwHk7O7Wcq1QUgAZxoneRcrB
S4IADRLO96UM7nM6rfF/dtBL8MWFVyK2BkyvZGALvXwk5P/MO3XGoU1rfrFxuXwu
sWBPidBnD4n+mJLVyC0e6hZn80edogOdHEjhGA9R08UKrnlv37Z+ri2poQFS1L9j
AgMBAAGjggFJMIIBRTCBvwYDVR0RBIG3MIG0ghJyZWdpc3RyeS5yZWRoYXQuaW+C
E3JlZ2lzdHJ5Ni5yZWRoYXQuaW+CHHJlZ2lzdHJ5Ni5jb25uZWN0LnJlZGhhdC5j
b22CG3JlZ2lzdHJ5Ni5hY2Nlc3MucmVkaGF0LmNvbYIbcmVnaXN0cnkuY29ubmVj
dC5yZWRoYXQuY29tghpyZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbYIVYWNjZXNz
LmNkbi5yZWRoYXQuY29tMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEF
BQcDAQYIKwYBBQUHAwIwCQYDVR0TBAIwADBHBgNVHR8EQDA+MDygOqA4hjZodHRw
Oi8vZ2F0ZXdheS56c2NhbGVydGhyZWUubmV0L3pzY2FsZXItenNjcmwtLTQtMS5j
cmwwDQYJKoZIhvcNAQELBQADggEBAHjHYL/F4WwkCWTzDgbvV8egx4WSauEU3gyF
pz++BXDsKYhGTWYz9qk/VowLFmpo9dc6lI9WvT6RmdOvinqRmZSif4rLs/016YEc
NevjDqTcKMExBJ0KUMYSzAqxPvdCqMLBeT1hzvtIYXGDTB34wKxUiUoRZhb/dit/
Us3fWpOFK/KC3kV9Fhgo2WyDhe1zTeozE5rZo+gUckEdzLbwwstlKhD2eGD0/a2a
Fa/n4rpnRFx6PNC0bekTfK72RtHLB/+VSRejciXCqZ6d0GfxYt//lQFFxSDvYiIN
LLxl9EUh4jnG1PXPalBrsqjS2pFYvwSkk54KbkWpFsv838h9XFk=
-----END CERTIFICATE-----
subject=C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", CN = registry.redhat.io
issuer=C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) "
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 4133 bytes and written 757 bytes
Verification error: unable to get local issuer certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 20 (unable to get local issuer certificate)
---
praveenkumar commented 1 year ago

This looks like you are using proxy or behind VPN and the connection is using the proxy. To me the issuer is DigCert and that is without proxy but for you it is CN = "Zscaler Intermediate Root CA (zscalerthree.net) (t) ". May be you need to check with admin and set the proxy for crc also.

$ openssl s_client -connect registry.redhat.io:443  2>/dev/null </dev/null | grep -i issuer
issuer=C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
cfergeau commented 1 year ago

Something like https://help.zscaler.com/zia/choosing-ca-certificate-ssl-inspection is probably enabled on your network.

santoshnt1 commented 1 year ago

Thanks for the help, I will try to find the Zscaler proxy and add it in crc config and check if it works Regards Santosh

cfergeau commented 1 year ago

Maybe it's enough to follow these steps to add the required CA to your cluster once it's running https://docs.openshift.com/container-platform/4.10/security/certificates/updating-ca-bundle.html

praveenkumar commented 1 year ago

@santoshnt1 did it fix for you after applying the proxy stuff? Can we close it?

santoshnt1 commented 1 year ago

Hi Praveen,

Yes after applying Christophe Fergeau solution it is working now . Thanks for your help. Best regards Santosh Kumar