search

crc-org / crc

CRC is a tool to help you run containers. It manages a local OpenShift 4.x cluster, Microshift or a Podman VM optimized for testing and development purposes
https://crc.dev
Apache License 2.0
1.26k stars 241 forks source link

crc 1.31.2 installation is failing on RHEL7; #2677

Closed rupeshad81 closed 3 years ago

rupeshad81 commented 3 years ago

After running the command : crc config set skip-check-daemon-systemd-unit true and crc config set skip-check-daemon-systemd-sockets true

It had proceeded further , but it is not able to do crc start.

[crc@eaas-crc ~]$ crc start --cpus 12 --memory 40000 --log-level debug
DEBU HTTP-PROXY: http://10.232.233.70:8080, HTTPS-PROXY: http://10.232.233.70:8080, NO-PROXY: 127.0.0.1,localhost,127.0.0.1,localhost,.corp.amdocs.com,.apps-crc.testing,.crc.testing,api-int.crc.testing, proxyCAFile:
DEBU CodeReady Containers version: 1.31.2+19538dab
DEBU OpenShift version: 4.8.4 (embedded in executable)
DEBU Running 'crc start'
DEBU Total memory of system is 42020716544 bytes
DEBU No new version available. The latest version is 1.31.2
DEBU Found binary path at /home/crc/.crc/bin/crc-driver-libvirt
DEBU Launching plugin server for driver libvirt
DEBU Plugin server listening at address 127.0.0.1:33893
DEBU () Calling .GetVersion
DEBU Using API Version 1
DEBU () Calling .SetConfigRaw
DEBU () Calling .GetMachineName
DEBU (crc) Calling .GetState
DEBU (crc) DBG | time="2021-08-27T08:24:38+01:00" level=debug msg="Getting current state..."
DEBU (crc) DBG | time="2021-08-27T08:24:38+01:00" level=debug msg="Fetching VM..."
DEBU Making call to close driver server
DEBU (crc) Calling .Close
DEBU Successfully made call to close driver server
DEBU Making call to close connection to plugin binary
DEBU (crc) DBG | time="2021-08-27T08:24:38+01:00" level=debug msg="Closing plugin on server side"
DEBU Checking if systemd-resolved.service is running
DEBU Running 'systemctl status systemd-resolved.service'
DEBU Command failed: exit status 4
DEBU stdout:
DEBU stderr: Unit systemd-resolved.service could not be found.
INFO Checking if running as non-root
INFO Checking if running inside WSL2
INFO Checking if crc-admin-helper executable is cached
DEBU Running '/home/crc/.crc/bin/crc-admin-helper-linux --version'
DEBU Found crc-admin-helper-linux version 0.0.6
DEBU crc-admin-helper executable already cached
INFO Checking for obsolete admin-helper executable
DEBU Checking if an older admin-helper executable is installed
DEBU No older admin-helper executable found
INFO Checking if running on a supported CPU architecture
INFO Checking minimum RAM requirements
DEBU Total memory of system is 42020716544 bytes
INFO Checking if crc executable symlink exists
INFO Checking if Virtualization is enabled
DEBU Checking if the vmx/svm flags are present in /proc/cpuinfo
DEBU CPU virtualization flags are good
INFO Checking if KVM is enabled
DEBU Checking if /dev/kvm exists
DEBU /dev/kvm was found
INFO Checking if libvirt is installed
DEBU Checking if 'virsh' is available
DEBU 'virsh' was found in /usr/bin/virsh
DEBU Checking 'virsh capabilities' for libvirtd/qemu availability
DEBU Running 'virsh --readonly --connect qemu:///system capabilities'
DEBU Found x86_64 hypervisor with 'hvm' capabilities
INFO Checking if user is part of libvirt group
DEBU Checking if current user is part of the libvirt group
DEBU Current user is already in the libvirt group
INFO Checking if active user/process is currently part of the libvirt group
INFO Checking if libvirt daemon is running
DEBU Checking if libvirtd service is running
DEBU Running 'systemctl status virtqemud.socket'
DEBU Command failed: exit status 3
DEBU stdout: * virtqemud.socket
   Loaded: masked (/dev/null; bad)
   Active: inactive (dead)
DEBU stderr:
DEBU virtqemud.socket is neither running nor listening
DEBU Running 'systemctl status libvirtd.socket'
DEBU Command failed: exit status 4
DEBU stdout:
DEBU stderr: Unit libvirtd.socket could not be found.
DEBU libvirtd.socket is neither running nor listening
DEBU Running 'systemctl status virtqemud.service'
DEBU Command failed: exit status 4
DEBU stdout:
DEBU stderr: Unit virtqemud.service could not be found.
DEBU virtqemud.service is neither running nor listening
DEBU Running 'systemctl status libvirtd.service'
DEBU libvirtd.service is running
INFO Checking if a supported libvirt version is installed
DEBU Checking if libvirt version is >=3.4.0
DEBU Running 'virsh -v'
INFO Checking if crc-driver-libvirt is installed
DEBU Checking if crc-driver-libvirt is installed
DEBU Running '/home/crc/.crc/bin/crc-driver-libvirt version'
DEBU Found crc-driver-libvirt version 0.13.1
DEBU crc-driver-libvirt is already installed
INFO Checking crc daemon systemd socket units
WARN Skipping above check...
INFO Checking if systemd-networkd is running
DEBU Checking if systemd-networkd.service is running
DEBU Running 'systemctl status systemd-networkd.service'
DEBU Command failed: exit status 4
DEBU stdout:
DEBU stderr: Unit systemd-networkd.service could not be found.
DEBU systemd-networkd.service is not running
INFO Checking if NetworkManager is installed
DEBU Checking if 'nmcli' is available
DEBU 'nmcli' was found in /usr/bin/nmcli
INFO Checking if NetworkManager service is running
DEBU Checking if NetworkManager.service is running
DEBU Running 'systemctl status NetworkManager.service'
DEBU NetworkManager.service is already running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
DEBU Checking NetworkManager configuration
DEBU NetworkManager configuration is good
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
DEBU Checking dnsmasq configuration
DEBU dnsmasq configuration is good
INFO Checking if libvirt 'crc' network is available
DEBU Checking if libvirt 'crc' network exists
DEBU Running 'virsh --connect qemu:///system net-info crc'
DEBU Checking if libvirt 'crc' definition is up to date
DEBU Running 'virsh --connect qemu:///system net-dumpxml --inactive crc'
DEBU libvirt 'crc' network has the expected value
INFO Checking if libvirt 'crc' network is active
DEBU Checking if libvirt 'crc' network is active
DEBU Running 'virsh --connect qemu:///system net-info crc'
DEBU libvirt 'crc' network is already active
DEBU Checking file: /home/crc/.crc/machines/crc/.crc-exist
DEBU Found binary path at /home/crc/.crc/bin/crc-driver-libvirt
DEBU Launching plugin server for driver libvirt
DEBU Plugin server listening at address 127.0.0.1:39813
DEBU () Calling .GetVersion
DEBU Using API Version 1
DEBU () Calling .SetConfigRaw
DEBU () Calling .GetMachineName
DEBU (crc) Calling .GetBundleName
DEBU (crc) Calling .GetState
DEBU (crc) DBG | time="2021-08-27T08:24:38+01:00" level=debug msg="Getting current state..."
DEBU (crc) DBG | time="2021-08-27T08:24:38+01:00" level=debug msg="Fetching VM..."
INFO Starting CodeReady Containers VM for OpenShift 4.8.4...
DEBU Updating CRC VM configuration
DEBU (crc) Calling .GetConfigRaw
DEBU (crc) Calling .Start
DEBU (crc) DBG | time="2021-08-27T08:24:38+01:00" level=debug msg="Starting VM crc"
DEBU (crc) DBG | time="2021-08-27T08:24:38+01:00" level=debug msg="Validating network"
DEBU (crc) DBG | time="2021-08-27T08:24:38+01:00" level=debug msg="Validating storage pool"
DEBU (crc) DBG | time="2021-08-27T08:24:44+01:00" level=debug msg="GetIP called for crc"
DEBU (crc) DBG | time="2021-08-27T08:24:44+01:00" level=debug msg="Getting current state..."
DEBU (crc) DBG | time="2021-08-27T08:24:44+01:00" level=debug msg="IP address: 192.168.130.11"
DEBU (crc) DBG | time="2021-08-27T08:24:44+01:00" level=info msg="Found IP for machine: 192.168.130.11"
DEBU (crc) Calling .GetConfigRaw
DEBU Waiting for machine to be running, this may take a few minutes...
DEBU retry loop: attempt 0
DEBU (crc) Calling .GetState
DEBU (crc) DBG | time="2021-08-27T08:24:44+01:00" level=debug msg="Getting current state..."
DEBU Machine is up and running!
DEBU (crc) Calling .GetState
DEBU (crc) DBG | time="2021-08-27T08:24:44+01:00" level=debug msg="Getting current state..."
DEBU (crc) Calling .GetIP
DEBU (crc) DBG | time="2021-08-27T08:24:44+01:00" level=debug msg="GetIP called for crc"
DEBU (crc) DBG | time="2021-08-27T08:24:44+01:00" level=debug msg="Getting current state..."
DEBU (crc) DBG | time="2021-08-27T08:24:44+01:00" level=debug msg="IP address: 192.168.130.11"
INFO CodeReady Containers instance is running with IP 192.168.130.11
DEBU Waiting until ssh is available
DEBU retry loop: attempt 0
DEBU Running SSH command: exit 0
DEBU Using ssh private keys: [/home/crc/.crc/cache/crc_libvirt_4.8.4/id_ecdsa_crc /home/crc/.crc/machines/crc/id_ecdsa]
DEBU SSH command results: err: dial tcp 192.168.130.11:22: i/o timeout, output:
DEBU error: Temporary error: ssh command error:
command : exit 0
err     : dial tcp 192.168.130.11:22: i/o timeout\n - sleeping 1s
...
DEBU retry loop: attempt 91
DEBU Running SSH command: exit 0
DEBU Using ssh private keys: [/home/crc/.crc/cache/crc_libvirt_4.8.4/id_ecdsa_crc /home/crc/.crc/machines/crc/id_ecdsa]
DEBU SSH command results: err: dial tcp 192.168.130.11:22: connect: no route to host, output:
DEBU error: Temporary error: ssh command error:
command : exit 0
err     : dial tcp 192.168.130.11:22: connect: no route to host\n - sleeping 1s
DEBU RetryAfter timeout after 92 tries
DEBU Making call to close driver server
DEBU (crc) Calling .Close
DEBU Successfully made call to close driver server
DEBU (crc) DBG | time="2021-08-27T08:29:45+01:00" level=debug msg="Closing plugin on server side"
DEBU Making call to close connection to plugin binary
Failed to connect to the CRC VM with SSH -- host might be unreachable: Temporary error: ssh command error:
command : exit 0
err     : dial tcp 192.168.130.11:22: i/o timeout\n (x3)
Temporary error: ssh command error:
command : exit 0
err     : dial tcp 192.168.130.11:22: connect: no route to host\n (x89)
[crc@eaas-crc ~]$ crc status
CRC VM:          Running
OpenShift:       Degraded (v4.8.4)
Disk Usage:      6.279GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.15GB
Cache Directory: /home/crc/.crc/cache
gbraad commented 3 years ago

DEBU SSH command results: err: dial tcp 192.168.130.11:22: connect: no route to host, output:

Issues with networking. Can you provide the information as asked in the issue template?

cfergeau commented 3 years ago

Can you check virsh -c qemu:///system list --all?

rupeshad81 commented 3 years ago

Also one point , i am using same ca-bundle configmap ,which used for older version , so is it ok? Or we had any new for 4.8.4

After then we apply 

oc apply -f /tmp/amdocs-cert.yaml
oc patch proxy cluster --type merge -p '{"spec":{"trustedCA":{"name":"user-ca-bundle"}}}'

Create certificate for amdocs CA

scp -i ~/.crc/machines/crc/id_ecdsa -o StrictHostKeyChecking=no /tmp/amdocs-cert.crt core@$(crc ip):/tmp

Copy to the correct on the VM and apply.

ssh -i ~/.crc/machines/crc/id_ecdsa -o StrictHostKeyChecking=no core@$(crc ip) sudo cp /tmp/amdocs-cert.crt /etc/pki/ca-trust/source/anchors/openshift-config-user-ca-bundle.crt
ssh -i ~/.crc/machines/crc/id_ecdsa -o StrictHostKeyChecking=no core@$(crc ip) sudo update-ca-trust
ssh -i ~/.crc/machines/crc/id_ecdsa -o StrictHostKeyChecking=no core@$(crc ip) sudo systemctl restart crio
ssh -i ~/.crc/machines/crc/id_ecdsa -o StrictHostKeyChecking=no core@$(crc ip) sudo systemctl restart kubelet
rupeshad81 commented 3 years ago 
[crc@eaas-crc ~]$ crc status
CRC VM:          Running
OpenShift:       Unreachable (v4.8.4)
Disk Usage:      5.836GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.15GB
Cache Directory: /home/crc/.crc/cache
[crc@eaas-crc ~]$ virsh -c qemu:///system list --all
 Id    Name                           State
----------------------------------------------------
 1     crc                            running
gbraad commented 3 years ago

I am not really following what the issue is now. You posted an initial issue that shows networking issue, but apply changes using ssh. Did these ever succeed?

Please follow the issue template and state clearly steps performed and the output of each.

DEBU retry loop: attempt x
DEBU Running SSH command: exit 0
DEBU Using ssh private keys: [/home/crc/.crc/cache/crc_libvirt_4.8.4/id_ecdsa_crc /home/crc/.crc/machines/crc/id_ecdsa]
DEBU SSH command results: err: dial tcp 192.168.130.11:22: i/o timeout, output:
DEBU error: Temporary error: ssh command error:
command : exit 0
err     : dial tcp 192.168.130.11:22: i/o timeout\n - sleeping 1s

This would mean the mentioned ssh commands in https://github.com/code-ready/crc/issues/2677#issuecomment-907088726 could never run successfully. Right? Or is the shown output from a restart of the same VM?

rupeshad81 commented 3 years ago

Mainly the crc status is showing the same status:

[crc@eaas-crc ~]$  crc status
CRC VM:          Running
OpenShift:       Degraded (v4.8.4)
Disk Usage:      6.405GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.15GB
Cache Directory: /home/crc/.crc/cache

So i had shared the steps, what i had followed for installation.

gbraad commented 3 years ago

So i had shared the steps, what i had followed for installation.

I am confused about your order of execution. Did the ssh commands return success? If so, this means the VM would have an IP address... ... but this is not reflected in the VM/CRC startup, as it states no route to host.

Can you perform a clean start:

$ crc delete
$ crc start

and indicate what the output is.

Note; removed statement about daemon process as system mode networking is used.

rupeshad81 commented 3 years ago

Hi, Please check now:

[crc@eaas-crc ~]$  crc status
CRC VM:          Running
OpenShift:       Degraded (v4.8.4)
Disk Usage:      6.405GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.15GB
Cache Directory: /home/crc/.crc/cache
[crc@eaas-crc ~]$ crc status
CRC VM:          Running
OpenShift:       Degraded (v4.8.4)
Disk Usage:      6.427GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.15GB
Cache Directory: /home/crc/.crc/cache
[crc@eaas-crc ~]$ crc delete
Do you want to delete the OpenShift cluster? [y/N]: y
WARN open /home/crc/.crc/machines/crc/kubeconfig: no such file or directory
Deleted the OpenShift cluster
[crc@eaas-crc ~]$ crc start --cpus 12 --memory 40000
INFO Checking if running as non-root
INFO Checking if running inside WSL2
INFO Checking if crc-admin-helper executable is cached
INFO Checking for obsolete admin-helper executable
INFO Checking if running on a supported CPU architecture
INFO Checking minimum RAM requirements
INFO Checking if crc executable symlink exists
INFO Checking if Virtualization is enabled
INFO Checking if KVM is enabled
INFO Checking if libvirt is installed
INFO Checking if user is part of libvirt group
INFO Checking if active user/process is currently part of the libvirt group
INFO Checking if libvirt daemon is running
INFO Checking if a supported libvirt version is installed
INFO Checking if crc-driver-libvirt is installed
INFO Checking crc daemon systemd socket units
WARN Skipping above check...
INFO Checking if systemd-networkd is running
INFO Checking if NetworkManager is installed
INFO Checking if NetworkManager service is running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
INFO Checking if libvirt 'crc' network is available
INFO Checking if libvirt 'crc' network is active
CodeReady Containers requires a pull secret to download content from Red Hat.
You can copy it from the Pull Secret section of https://cloud.redhat.com/openshift/create/local.
? Please enter the pull secret ******************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
WARN Cannot add pull secret to keyring: The name org.freedesktop.secrets was not provided by any .service files
INFO Loading bundle: crc_libvirt_4.8.4...
INFO Creating CodeReady Containers VM for OpenShift 4.8.4...
INFO Generating new SSH Key pair...
INFO Generating new password for the kubeadmin user
INFO Starting CodeReady Containers VM for OpenShift 4.8.4...
INFO CodeReady Containers instance is running with IP 192.168.130.11
INFO CodeReady Containers VM is running
INFO Updating authorized keys...
INFO Check internal and public DNS query...
INFO Check DNS query from host...
INFO Adding proxy configuration to kubelet and crio service...
INFO Verifying validity of the kubelet certificates...
INFO Starting OpenShift kubelet service
INFO Waiting for kube-apiserver availability... [takes around 2min]
INFO Adding user's pull secret to the cluster...
INFO Updating SSH key to machine config resource...
INFO Waiting for user's pull secret part of instance disk...
Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x11)
[crc@eaas-crc ~]$ crc status
CRC VM:          Running
OpenShift:       Unreachable (v4.8.4)
Disk Usage:      11.68GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.15GB
Cache Directory: /home/crc/.crc/cache
[crc@eaas-crc ~]$ crc daemon
INFO Checking if running as non-root
INFO Checking if running inside WSL2
INFO Checking if crc-admin-helper executable is cached
INFO Checking for obsolete admin-helper executable
INFO Checking if running on a supported CPU architecture
INFO Checking minimum RAM requirements
INFO Checking if crc executable symlink exists
INFO Checking if Virtualization is enabled
INFO Checking if KVM is enabled
INFO Checking if libvirt is installed
INFO Checking if user is part of libvirt group
INFO Checking if active user/process is currently part of the libvirt group
INFO Checking if libvirt daemon is running
INFO Checking if a supported libvirt version is installed
INFO Checking if crc-driver-libvirt is installed
INFO Checking crc daemon systemd socket units
WARN Skipping above check...
INFO Checking if systemd-networkd is running
INFO Checking if NetworkManager is installed
INFO Checking if NetworkManager service is running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
INFO Checking if libvirt 'crc' network is available
INFO Checking if libvirt 'crc' network is active
INFO listening vsock://:1024
listen vsock: open /dev/vsock: permission denied
[crc@eaas-crc ~]$ crc status
CRC VM:          Running
OpenShift:       Unreachable (v4.8.4)
Disk Usage:      11.71GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.15GB
Cache Directory: /home/crc/.crc/cache

Thanks

gbraad commented 3 years ago

Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x11)

can you please start from a clean setup. I am confused about the steps you perform.

$ crc cleanup
$ crc setup
$ crc start
rupeshad81 commented 3 years ago

Hi,

Same procedure was done and updated above comments.

[crc@eaas-crc ~]$ crc delete
Do you want to delete the OpenShift cluster? [y/N]: y
WARN open /home/crc/.crc/machines/crc/kubeconfig: no such file or directory
Deleted the OpenShift cluster

[crc@eaas-crc ~]$ crc start --cpus 12 --memory 40000
INFO Checking if running as non-root
INFO Checking if running inside WSL2
INFO Checking if crc-admin-helper executable is cached
INFO Checking for obsolete admin-helper executable
INFO Checking if running on a supported CPU architecture
INFO Checking minimum RAM requirements
INFO Checking if crc executable symlink exists
INFO Checking if Virtualization is enabled
INFO Checking if KVM is enabled
INFO Checking if libvirt is installed
INFO Checking if user is part of libvirt group
INFO Checking if active user/process is currently part of the libvirt group
INFO Checking if libvirt daemon is running
INFO Checking if a supported libvirt version is installed
INFO Checking if crc-driver-libvirt is installed
INFO Checking crc daemon systemd socket units
WARN Skipping above check...
INFO Checking if systemd-networkd is running
INFO Checking if NetworkManager is installed
INFO Checking if NetworkManager service is running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
INFO Checking if libvirt 'crc' network is available
INFO Checking if libvirt 'crc' network is active
CodeReady Containers requires a pull secret to download content from Red Hat.
You can copy it from the Pull Secret section of https://cloud.redhat.com/openshift/create/local.
? Please enter the pull secret ******************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
WARN Cannot add pull secret to keyring: The name org.freedesktop.secrets was not provided by any .service files
INFO Loading bundle: crc_libvirt_4.8.4...
INFO Creating CodeReady Containers VM for OpenShift 4.8.4...
INFO Generating new SSH Key pair...
INFO Generating new password for the kubeadmin user
INFO Starting CodeReady Containers VM for OpenShift 4.8.4...
INFO CodeReady Containers instance is running with IP 192.168.130.11
INFO CodeReady Containers VM is running
INFO Updating authorized keys...
INFO Check internal and public DNS query...
INFO Check DNS query from host...
INFO Adding proxy configuration to kubelet and crio service...
INFO Verifying validity of the kubelet certificates...
INFO Starting OpenShift kubelet service
INFO Waiting for kube-apiserver availability... [takes around 2min]
INFO Adding user's pull secret to the cluster...
INFO Updating SSH key to machine config resource...
INFO Waiting for user's pull secret part of instance disk...
Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x11)
[crc@eaas-crc ~]$ crc status
CRC VM:          Running
OpenShift:       Unreachable (v4.8.4)
Disk Usage:      11.68GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.15GB
Cache Directory: /home/crc/.crc/cache

[crc@eaas-crc ~]$ crc daemon
INFO Checking if running as non-root
INFO Checking if running inside WSL2
INFO Checking if crc-admin-helper executable is cached
INFO Checking for obsolete admin-helper executable
INFO Checking if running on a supported CPU architecture
INFO Checking minimum RAM requirements
INFO Checking if crc executable symlink exists
INFO Checking if Virtualization is enabled
INFO Checking if KVM is enabled
INFO Checking if libvirt is installed
INFO Checking if user is part of libvirt group
INFO Checking if active user/process is currently part of the libvirt group
INFO Checking if libvirt daemon is running
INFO Checking if a supported libvirt version is installed
INFO Checking if crc-driver-libvirt is installed
INFO Checking crc daemon systemd socket units
WARN Skipping above check...
INFO Checking if systemd-networkd is running
INFO Checking if NetworkManager is installed
INFO Checking if NetworkManager service is running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
INFO Checking if libvirt 'crc' network is available
INFO Checking if libvirt 'crc' network is active
INFO listening vsock://:1024
listen vsock: open /dev/vsock: permission denied

[crc@eaas-crc ~]$ crc status
CRC VM:          Running
OpenShift:       Unreachable (v4.8.4)
Disk Usage:      11.71GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.15GB
Cache Directory: /home/crc/.crc/cache
cfergeau commented 3 years ago

Note: you do not need to run crc daemon

rupeshad81 commented 3 years ago

Hi,

crc daemon , as per your request to see the output.

Thanks

gbraad commented 3 years ago

That was 'my' request and was in error. I had removed the statement. You are running in system mode for networking. It does however not change anything, as I also asked to follow the issue template, let me know if the ssh command for the CA certs was successful, etc but without this information it is hard to assess where things went wrong, as you make it sound it allows you to manuallly run ssh commands, which is very unlikely.

gbraad commented 3 years ago

Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x11)

Which is weird, and likely one of the reasons the cluster does not work well. What is in your crc config view ? (as requested per issue template).

It might be advised to store the pull secret on disk and use the pull-secret-path as config option to point to this file.

gbraad commented 3 years ago

Can you at least answer the following?

General information

Host Operating System

# Put the output of `cat /etc/os-release` in case of Linux
# put the output of `sw_vers` in case of Mac
# Put the output of `systeminfo` in case of Windows
rupeshad81 commented 3 years ago

Hi, Please check below: [crc@eaas-crc ~]$ cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.8 (Maipo)

Thanks

gbraad commented 3 years ago

Laptop, baremetal or a Virtual Machine?

rupeshad81 commented 3 years ago

Hi, It is Virtual Machine.

Thanks

gbraad commented 3 years ago

Which I guessed. It shows erratic behavior. We do not support nested virtualization setups. While it is known to work with KVM and HyperV, some unexpected behavior has been reported. Especially in combination with VMware. We are unable to verify and test this. All you can do is gather logs and provide detailed information, but we are unable to resolve this. Try to run this without a nested setup and verify your installation works.

rupeshad81 commented 3 years ago

Hi,

We were able to install till 4.7.11 on the same VM , without any issues. How it is changed from 4.8.x , any changes in setup, we should have some workaround. Or else Redhat should provide the proper setup , which is compatible for VM. same way like 4.7.11.

So let us know the findings.

Thanks

gbraad commented 3 years ago

The single node setup is not a supported setup and is only offered for development purposes in a best effort approach. We have already experienced a lot of issues just delivering 4.8 as is for this purpose as according to #2509. If you need to run a supported setup, please use the OpenShift installer.

Addition, we support native hypervisors, like KVM, HyperV and Hyperkit, as we are able to run tests for these. Anything else for the snc setup is an as-is deliverable. YMMV.

rupeshad81 commented 3 years ago

Hi,

For development purpose we need this setup, as single node. Further which customer will deploy their deployments on the cluster levels and then production clusters.

But this get failed with various different errors , so it is very difficult to troubleshoot this errors.

Thanks

gbraad commented 3 years ago

we should have some workaround.

Use the OpenShift installer to setup an environment using VMware as the hypervisor of your choice. You can 'follow' (read: learn from) how to run a single node setup from the code-ready/snc repo. This will not be a one to one mapping, but would avoid any possible issues caused by a nested virtualization setup.

rupeshad81 commented 3 years ago

Hi,

Please can you share the proper steps to follow and instruction to run the setup.

Thanks

gbraad commented 3 years ago

We are not able to do so. The snc repo can provide some guidance, but since this concerns VMware we could not verify any of the steps.

You would use the installer with a forced single master and worker setup, but allow to run this converged on a single node.

I can not guarantee this will even work, as we only test this setup on KVM, and using gcp. Only recently 4.8 started work on a single node setup for edge purposes, it it is still early days. So, your mileage may vary.

rupeshad81 commented 3 years ago

Hi,

When there will be stable CRC version 4.8.x to installed it.

Thanks

gbraad commented 3 years ago

CRC 4.8.x is stable for the intended use of installing on baremetal and/or laptop use, as these are our primary targets.

For use on nested virtualization your mileage may vary; tested to work on KVM+HyperV, and HyperV+KVM, HyperV+HyperV and KVM+KVM. Any other combination involving a hypervisor other than these can result in unexpected results.

For any other setup, please consider using the OpenShift installer: https://github.com/openshift/installer

gbraad commented 3 years ago

Note: we might investigate this over time, but we have higher priorities issues than making this work in combination with VMware. Closing as a duplicate of #2628

cfergeau commented 3 years ago

If you can consistently reproduce the Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x11) error after crc cleanup && crc setup && crc start --log-level debug (only these 3 commands with no additional steps or arguments), I'd be interested in the output of crc start with debugging output.

rupeshad81 commented 3 years ago

Hi, For now we had removed the setup and installing the crc 1.29.1 with ocp version 4.7.18.

But getting this issue for the same version.

gbraad commented 3 years ago

But getting this issue for the same version.

Which issue ?

rupeshad81 commented 3 years ago

Hi,

Failed to renew TLS certificates => 1.29.1

Both for 1.31.1 and 1.31.2 no route to host\n (x89) pull secret not updated to disk (x11)

gbraad commented 3 years ago

the pull-secret not updating and no route to host is most likely related to not getting a connection established.

the failed TLS certificate is a timing sensitive process, so nested virtualization can interfere here. this has been reported before. so in that case, not much that can be done.