[BUG] Error certificate has expired or is not yet valid, when first time installing crc on Windows 11 Pro

[ahmadsyahidr]

General information

• OS: Microsoft Windows 11 Pro
• Hypervisor: Hyper-V
• Did you run crc setup before starting it ? Yes
• Running CRC on: Laptop

CRC version

CRC version: 2.18.0+4ea3a1
OpenShift version: 4.12.13
Podman version: 4.4.1

CRC status

CRC VM:          Running
OpenShift:       Running (v4.12.13)
RAM Usage:       6.654GB of 9.399GB
Disk Usage:      16.97GB of 32.74GB (Inside the CRC VM)
Cache Usage:     19.71GB
Cache Directory: C:\Users\ahmad\.crc\cache

CRC config

- consent-telemetry                     : yes

Host Operating System

Host Name:                 MSI-GAMING-GF63
OS Name:                   Microsoft Windows 11 Pro
OS Version:                10.0.22621 N/A Build 22621
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:          ahmad_syahid@outlook.com
Registered Organization:
Product ID:                00330-80000-00000-AA807
Original Install Date:     14/07/2022, 18:48:59
System Boot Time:          04/05/2023, 16:21:23
System Manufacturer:       Micro-Star International Co., Ltd.
System Model:              GF63 Thin 9RCX
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 158 Stepping 10 GenuineIntel ~2400 Mhz
BIOS Version:              American Megatrends Inc. E16R3IMS.309, 26/03/2020
Windows Directory:         C:\WINDOWS
System Directory:          C:\WINDOWS\system32
Boot Device:               \Device\HarddiskVolume3
System Locale:             en-gb;English (United Kingdom)
Input Locale:              en-gb;English (United Kingdom)
Time Zone:                 (UTC+07:00) Bangkok, Hanoi, Jakarta
Total Physical Memory:     32.611 MB
Available Physical Memory: 20.972 MB
Virtual Memory: Max Size:  34.659 MB
Virtual Memory: Available: 19.876 MB
Virtual Memory: In Use:    14.783 MB
Page File Location(s):     C:\pagefile.sys
Domain:                    WORKGROUP
Logon Server:              \\MSI-GAMING-GF63
Hotfix(s):                 4 Hotfix(s) Installed.
                           [01]: KB5025182
                           [02]: KB5012170
                           [03]: KB5025305
                           [04]: KB5025351
Network Card(s):           8 NIC(s) Installed.
                           [01]: TAP-ProtonVPN Windows Adapter V9
                                 Connection Name: Local Area Connection
                                 Status:          Media disconnected
                           [02]: Intel(R) Wireless-AC 9560 160MHz
                                 Connection Name: WiFi
                                 DHCP Enabled:    Yes
                                 DHCP Server:     192.168.1.1
                                 IP address(es)
                                 [01]: 192.168.1.5
                           [03]: Fortinet Virtual Ethernet Adapter (NDIS 6.30)
                                 Connection Name: Ethernet 6
                                 Status:          Media disconnected
                           [04]: VMware Virtual Ethernet Adapter for VMnet1
                                 Connection Name: VMware Network Adapter VMnet1
                                 DHCP Enabled:    Yes
                                 DHCP Server:     192.168.153.254
                                 IP address(es)
                                 [01]: 192.168.153.1
                           [05]: VMware Virtual Ethernet Adapter for VMnet8
                                 Connection Name: VMware Network Adapter VMnet8
                                 DHCP Enabled:    Yes
                                 DHCP Server:     192.168.64.254
                                 IP address(es)
                                 [01]: 192.168.64.1
                           [06]: Microsoft Wi-Fi Direct Virtual Adapter
                                 Connection Name: Local Area Connection* 10
                                 DHCP Enabled:    No
                                 IP address(es)
                                 [01]: 192.168.137.1
                           [07]: Bluetooth Device (Personal Area Network)
                                 Connection Name: Bluetooth Network Connection
                                 Status:          Media disconnected
                           [08]: Fortinet SSL VPN Virtual Ethernet Adapter
                                 Connection Name: Ethernet 7
                                 Status:          Hardware not present
Hyper-V Requirements:      A hypervisor has been detected. Features required for Hyper-V will not be displayed.

Steps to reproduce

1. Ensure hyper-v is enabled --> https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v
2. Download crc-windows-amd64.msi
3. Double Click on crc-windows-amd64.msi and proceed to install, and Restart
4. Open Command Prompt (Not Administrator)
5. Run crc setup
6. Run crc start
7. After few minutes the error appear: ERRO Cannot update kubeconfig: x509: certificate has expired or is not yet valid: current time 2023-05-04T19:47:56+07:00 is after 2022-08-23T04:15:45Z

Expected

1. No Certificate Error Message
2. CRC Status both Running and Openshift Console can be Open.
3. oc login -u developer https://api.crc.testing:6443 Login Successfully

Actual

1. ERRO Cannot update kubeconfig: x509: certificate has expired or is not yet valid: current time 2023-05-04T19:47:56+07:00 is after 2022-08-23T04:15:45Z
2. CRC Status both Running and Openshift Console cannot be Open.
3. oc login -u developer https://api.crc.testing:6443 Error from server (InternalError): Internal error occurred: unexpected response: 404

Logs

https://gist.github.com/mii-syahid/986020794eab92c050c82dbae3d55848

Before gather the logs try following if that fix your issue

$ crc delete -f
$ crc cleanup
$ crc setup
$ crc start --log-level debug

Already tried, they aren't solving the issue. Please consider posting the output of crc start --log-level debug on http://gist.github.com/ and post the link in the issue.

[gbraad]

Time Zone: (UTC+07:00) Bangkok, Hanoi, Jakarta

I have tried this on my machine

Time Zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi

but it does not look like a timezone issue

[ahmadsyahidr]

Time Zone: (UTC+07:00) Bangkok, Hanoi, Jakarta

I have tried this on my machine

Time Zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi

but it does not look like a timezone issue

Is it OK in your Environment ? I also have golang, android studio, and docker desktop+k8s in my computer.

[adrianriobo]

Just tested on windows 11 pro fresh environment

OS Name:                   Microsoft Windows 11 Pro
OS Version:                10.0.22621 N/A Build 22621
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation

Got a cluster running for 30 mins without any issue

CRC VM:          Running
OpenShift:       Running (v4.12.13)
RAM Usage:       6.517GB of 9.397GB
Disk Usage:      16.65GB of 32.74GB (Inside the CRC VM)
Cache Usage:     19.71GB
Cache Directory: C:\Users\rhqp\.crc\cache

[gbraad]

maybe entries in your kube config that cause a conflict?

[ahmadsyahidr]

maybe entries in your kube config that cause a conflict?

I've tried to clean up everything inside .kube folder before installation, but the same error still appear.

The .kube folder should be in C:/Users/myuser/.kube right ?

[praveenkumar]

@mii-syahid make sure you didn't have KUBECONFIG env set or if it set then don't have expired entry for crc.testing domain. As per logs cluster started and running successful.

[ahmadsyahidr]

@mii-syahid make sure you didn't have KUBECONFIG env set or if it set then don't have expired entry for crc.testing domain. As per logs cluster started and running successful.

Sorry, I've check on the environment variables lists, there is no kubeconfig or anything like that. The weird thing is, the certificate information looks like owned by VMware:

Common Name: VMware Subject Alternative Names: Organization: Organization Unit: VMware Locality: Palo Alto State: Country: US Valid From: August 22, 2021 Valid To: August 22, 2022 Key Size: 2048 bit Serial Number: f0257a4ae83597d4

[ahmadsyahidr]

@mii-syahid make sure you didn't have KUBECONFIG env set or if it set then don't have expired entry for crc.testing domain. As per logs cluster started and running successful.

Sorry, I've check on the environment variables lists, there is no kubeconfig or anything like that. The weird thing is, the certificate information looks like owned by VMware:

Common Name: VMware Subject Alternative Names: Organization: Organization Unit: VMware Locality: Palo Alto State: Country: US Valid From: August 22, 2021 Valid To: August 22, 2022 Key Size: 2048 bit Serial Number: f0257a4ae83597d4

Is it because I've installed crc on my VMware Workstation in my Laptop a long time ago ? VMware app was still there, only I can't open any vm because Hyper-v is up.

[ahmadsyahidr]

@mii-syahid make sure you didn't have KUBECONFIG env set or if it set then don't have expired entry for crc.testing domain. As per logs cluster started and running successful.

I've also check on C:\Users\ahmad.crc\cache\crc_hyperv_4.12.13_amd64\kubeconfig All certificate in there is good, nothing expired. I also add this path C:\Users\ahmad.crc\bin\oc in environment variable, to ensure I use the correct oc client.

oc version Client Version: 4.12.13 Kustomize Version: v4.5.7

Can you explain which certificate are this crc-start error are referring to ?

[cfergeau]

The weird thing is, the certificate information looks like owned by VMware:

Where did you get this certificate information from? Not clear why crc would end up trying to use this certificate/CA.

[ahmadsyahidr]

The weird thing is, the certificate information looks like owned by VMware:

Where did you get this certificate information from? Not clear why crc would end up trying to use this certificate/CA.

I got that information from web browser, accessing this console --> https://console-openshift-console.apps-crc.testing/

[cfergeau]

Can you check what IP address https://console-openshift-console.apps-crc.testing resolves to? Just to be sure, you are not using nested virtualization? (Windows running in a VMWare virtual machine, and crc installation happening inside this Windows VM).

[ahmadsyahidr]

Can you check what IP address https://console-openshift-console.apps-crc.testing resolves to? Just to be sure, you are not using nested virtualization? (Windows running in a VMWare virtual machine, and crc installation happening inside this Windows VM).

Sure, it was 127.0.0.1.

ping console-openshift-console.apps-crc.testing

Pinging api.crc.testing [127.0.0.1] with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

I'm sure it wasn't nested virtualization, I installed crc on my windows, and We can see a new virtual machine named by crc exists in hyper-v manager.

[praveenkumar]

@ahmadsyahidr Can you try latest version of crc and see if that fix the issue?

[ahmadsyahidr]

@praveenkumar Sure, but I can only try it next week. I will update the result here.

[praveenkumar]

@ahmadsyahidr did you get chance to try out latest version?

[nikolajajac]

@ahmadsyahidr Did you ever resolve this? I'm facing the exactly same issue...

[cfergeau]

What crc version are you using?

[nikolajajac]

I resolved the issue for me. I also have VMware installed on my machine. The expired certificate is located at C:\ProgramData\VMware\SSL. It is used by the VMware Workstation Server Windows service. Stopping this service allowed the crc start to complete successfully. As I don't actively use VMware right now, this solution is good enough. I'm using crc version 2.34.1 (currently the latest one).