search

crc-org / crc-extension

Red Hat OpenShift Local Extension for integration of OpenShift Local clusters with Podman Desktop
Apache License 2.0
13 stars 16 forks source link

[BUG] libvirtd daemon preflight check keeps failing #186

Closed dgolovin closed 7 months ago

dgolovin commented 8 months ago

General information

CRC version

2.28.0

CRC status

crc status
Machine does not exist. Use 'crc start' to create it

CRC config

N/A

Steps to reproduce

  1. run crc setup
  2. run crc setup --check-only
  3. run echo $?

Expected

step 3 prints 0

Actual

step 3 prints 2 on linux

Logs

crc setup --check-only --log-level debug
DEBU CRC version: 2.28.0+e1f42d                   
DEBU OpenShift version: 4.13.14                   
DEBU Podman version: 4.4.4                        
DEBU Running 'crc setup'                          
INFO Using bundle path /home/eskimo/.crc/cache/crc_libvirt_4.13.14_amd64.crcbundle 
DEBU Checking if systemd-resolved.service is running 
DEBU Running 'systemctl status systemd-resolved.service' 
DEBU systemd-resolved.service is already running  
INFO Checking if running as non-root              
INFO Checking if running inside WSL2              
INFO Checking if crc-admin-helper executable is cached 
DEBU Running '/home/eskimo/.crc/bin/crc-admin-helper-linux --version' 
DEBU Found crc-admin-helper-linux version 0.0.12  
DEBU crc-admin-helper executable already cached   
INFO Checking if running on a supported CPU architecture 
DEBU GOARCH is amd64 GOOS is linux                
INFO Checking if crc executable symlink exists    
INFO Checking minimum RAM requirements            
DEBU Total memory of system is 66757939200 bytes  
INFO Checking if Virtualization is enabled        
DEBU Checking if the vmx/svm flags are present in /proc/cpuinfo 
DEBU CPU virtualization flags are good            
INFO Checking if KVM is enabled                   
DEBU Checking if /dev/kvm exists                  
DEBU /dev/kvm was found                           
INFO Checking if libvirt is installed             
DEBU Checking if 'virsh' is available             
DEBU 'virsh' was found in /usr/bin/virsh          
DEBU Checking 'virsh capabilities' for libvirtd/qemu availability 
DEBU Running 'virsh --readonly --connect qemu:///system capabilities' 
DEBU Found x86_64 hypervisor with 'hvm' capabilities 
INFO Checking if user is part of libvirt group    
DEBU Checking if current user is part of the libvirt group 
DEBU Current user is already in the libvirt group 
INFO Checking if active user/process is currently part of the libvirt group 
INFO Checking if libvirt daemon is running        
DEBU Checking if libvirtd service is running      
DEBU Running 'systemctl status virtqemud.socket'  
DEBU Command failed: exit status 3                
DEBU stdout: * virtqemud.socket - Libvirt qemu local socket
     Loaded: loaded (/usr/lib/systemd/system/virtqemud.socket; disabled; preset: disabled)
     Active: inactive (dead)
   Triggers: * virtqemud.service
     Listen: /run/libvirt/virtqemud-sock (Stream) 
DEBU stderr:                                      
DEBU virtqemud.socket is neither running nor listening 
DEBU Running 'systemctl status libvirtd.socket'   
DEBU libvirtd.socket is running                   
INFO Checking if a supported libvirt version is installed 
DEBU Checking if libvirt version is >=3.4.0       
DEBU Running 'virsh -v'                           
INFO Checking if crc-driver-libvirt is installed  
DEBU Checking if crc-driver-libvirt is installed  
DEBU Running '/home/eskimo/.crc/bin/crc-driver-libvirt version' 
DEBU Found crc-driver-libvirt version 0.13.6      
DEBU crc-driver-libvirt is already installed      
INFO Checking crc daemon systemd service          
DEBU Checking crc daemon systemd service          
DEBU Checking if crc-daemon.service is running    
DEBU Running 'systemctl --user status crc-daemon.service' 
DEBU crc-daemon.service is running                
DEBU crc-daemon.service is running                
crc-daemon.service is running
gbraad commented 8 months ago

Could you provide the --log-level debug output for this command? There is too little information to determine the actual problem. What about the distro/version of Linux? which steps shows an issue or do all steps succeed but however still return an error?

dgolovin commented 8 months ago

Fedora 38

dgolovin commented 8 months ago

I've update description.

gbraad commented 8 months ago

Libvirtd related:

INFO Checking if libvirt daemon is running        
DEBU Checking if libvirtd service is running      
DEBU Running 'systemctl status virtqemud.socket'  
DEBU Command failed: exit status 3                
DEBU stdout: * virtqemud.socket - Libvirt qemu local socket
     Loaded: loaded (/usr/lib/systemd/system/virtqemud.socket; disabled; preset: disabled)
     Active: inactive (dead)
   Triggers: * virtqemud.service
     Listen: /run/libvirt/virtqemud-sock (Stream) 
DEBU stderr:                                      
DEBU virtqemud.socket is neither running nor listening 
DEBU Running 'systemctl status libvirtd.socket'   
DEBU libvirtd.socket is running

I believe this got fixed in the upstream. Please make sure you are running a fully updated Fedora installation.

/cc: @cfergeau

dgolovin commented 8 months ago

I updates everything, it looks like latest release to me, am I wrong?

$ sudo dnf upgrade
Last metadata expiration check: 0:15:37 ago on Mon 30 Oct 2023 09:26:58 PM PDT.
Dependencies resolved.
Nothing to do.
Complete!
[eskimo@localhost ~]$ dnf check-update
Copr repo for PyCharm owned by phracek                               86 kB/s |  45 kB     00:00    
Docker CE Stable - x86_64                                            61 kB/s |  15 kB     00:00    
Fedora 38 - x86_64                                                   19 MB/s |  83 MB     00:04    
Fedora 38 openh264 (From Cisco) - x86_64                            2.4 kB/s | 2.5 kB     00:01    
Fedora Modular 38 - x86_64                                          1.5 MB/s | 2.8 MB     00:01    
Fedora 38 - x86_64 - Updates                                         13 MB/s |  34 MB     00:02    
Fedora Modular 38 - x86_64 - Updates                                1.5 MB/s | 2.1 MB     00:01    
google-chrome                                                        14 kB/s | 3.6 kB     00:00    
RPM Fusion for Fedora 38 - Free                                     1.4 MB/s | 693 kB     00:00    
RPM Fusion for Fedora 38 - Free - Updates                           418 kB/s | 319 kB     00:00    
RPM Fusion for Fedora 38 - Nonfree - NVIDIA Driver                   22 kB/s |  14 kB     00:00    
RPM Fusion for Fedora 38 - Nonfree - Steam                          2.0 kB/s | 2.2 kB     00:01    
slack                                                               8.4 kB/s | 6.4 kB     00:00    
[eskimo@localhost ~]$ dnf updateinfo list
Last metadata expiration check: 0:00:17 ago on Mon 30 Oct 2023 09:47:41 PM PDT.
[eskimo@localhost ~]$ dnf upgrade
Error: This command has to be run with superuser privileges (under the root user on most systems).
[eskimo@localhost ~]$ sudo dnf upgrade
[sudo] password for eskimo: 
Last metadata expiration check: 0:21:37 ago on Mon 30 Oct 2023 09:26:58 PM PDT.
Dependencies resolved.
Nothing to do.
Complete!
[eskimo@localhost ~]$ lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: Fedora
Description:    Fedora release 38 (Thirty Eight)
Release:    38
Codename:   ThirtyEight
[eskimo@localhost ~]$
praveenkumar commented 8 months ago

The overall preflight check is not failing, below is just failure of virtqemud socket activation which might not be active by default. https://libvirt.org/daemons.html#modular-driver-daemons check the all modular daemon/socket. I am hoping that after crc setup --check-only you have exit code 0 . 

INFO Checking if active user/process is currently part of the libvirt group 
INFO Checking if libvirt daemon is running        
DEBU Checking if libvirtd service is running      
DEBU Running 'systemctl status virtqemud.socket'  
DEBU Command failed: exit status 3                
DEBU stdout: * virtqemud.socket - Libvirt qemu local socket
     Loaded: loaded (/usr/lib/systemd/system/virtqemud.socket; disabled; preset: disabled)
     Active: inactive (dead)
   Triggers: * virtqemud.service
     Listen: /run/libvirt/virtqemud-sock (Stream) 
DEBU stderr:                                      
DEBU virtqemud.socket is neither running nor listening 
DEBU Running 'systemctl status libvirtd.socket'   
DEBU libvirtd.socket is running                   
INFO Checking if a supported libvirt version is installed
dgolovin commented 8 months ago

I have exit code 2 when crc instance is not running and 0 when it is running.

gbraad commented 8 months ago

In recent version of Fedora this uses socket activation, which might explain the initial error, but when it is running it shows 0. Our preflight is wrong in that case...

cfergeau commented 8 months ago

An error occurred during the checks, which is reported and the exit code is set to 2. The problem is the error reporting/logging, as the error is crc-daemon.service is running. If you run without --check-only, the next message will be DEBU Stopping crc-daemon.service

cfergeau commented 8 months ago

https://github.com/crc-org/crc/blob/9e31bb05f123b44904afd11a1033f3a1ea65e614/pkg/crc/preflight/preflight_checks_linux.go#L357-L362 or https://github.com/crc-org/crc/blob/9e31bb05f123b44904afd11a1033f3a1ea65e614/pkg/crc/preflight/preflight_checks_linux.go#L334-L336 can be reworked to make the error explicit.

cfergeau commented 8 months ago

We have this preflight check to ensure the daemon is restarted on upgrades.

cfergeau commented 8 months ago

Steps to reproduce

run crc setup run crc setup --check-only run echo $?

I can't reproduce with these steps on linux with 2.29.0, an intermediate step is missing I believe:

$ crc setup
# do something which will start the daemon, can be `crc start`, or just `curl --unix-socket ~/.crc/crc-http.sock  http://local/api/version`
$ crc setup --check-only
$ echo $?
cfergeau commented 8 months ago

Can you describe your high level usecase/requirement for --check-only? It's useful to have it in mind while we work on crc.

dgolovin commented 8 months ago

We need a consistent way to check that setup was executed and crc is ready to be started with 'crc start'. Right now 'crc setup' returns error code, but I still can do crc start without issues. So --check-only option gives false result in my case.