crc-org / crc

CRC is a tool to help you run containers. It manages a local OpenShift 4.x cluster, Microshift or a Podman VM optimized for testing and development purposes
https://crc.dev
Apache License 2.0
1.26k stars 242 forks source link

[BUG] Running Microshift instance disappeared #3754

Open deboer-tim opened 1 year ago

deboer-tim commented 1 year ago

General information

CRC version

WARN A new version (2.20.0) has been published on https://developers.redhat.com/content-gateway/file/pub/openshift-v4/clients/crc/2.20.0/crc-macos-installer.pkg 
CRC version: 2.17.0+44e15711
OpenShift version: 4.12.9
Podman version: 4.4.1

CRC status

DEBU CRC version: 2.17.0+44e15711                 
DEBU OpenShift version: 4.12.9                    
DEBU Podman version: 4.4.1                        
DEBU Running 'crc status'                         
DEBU Running 'sw_vers -productVersion'            
crc does not seem to be setup correctly, have you run 'crc setup'?

CRC config

- consent-telemetry                     : yes
- cpus                                  : 4
- memory                                : 16310
- preset                                : microshift

Host Operating System

# Put the output of `cat /etc/os-release` in case of Linux
# put the output of `sw_vers` in case of Mac
# Put the output of `systeminfo` in case of Windows

Steps to reproduce

Unknown.

Expected

I left Microshift running last night, stopped Podman Desktop, and likely opened and closed my laptop a couple times as I was logging off. When I logged in this morning and restarted Podman Desktop I would expect Microshift to still be running fine.

Actual

My instance is gone from within Podman Desktop. crc status thinks I haven't even run setup yet.

I know I can recreate the instance, but is there anything I can do to help figure out what stopped/deleted it?

gbraad commented 1 year ago

We have seen reports of hibernation killing the vfkit process. Will confirm this with @cfergeau. It is driver issue that has been filed before for crc.

gbraad commented 1 year ago

2.17.0+44e15711

kinda old. it should not affect the driver, but please consider updating. (Note: the version of Microshift it provides is known to be unstable according to our tests. 4.13 does not have the reported issue).

deboer-tim commented 1 year ago

As per the call we had earlier, I didn't know it wasn't auto-updating (the cache issue :) ). I will upgrade, just wanted to confirm if there were any logs or other things you wanted to look at before I touch them.

We have seen reports of hibernation killing the vfkit process

If vfkit forced the instance to stop it wouldn't be great, but at least I could just start it again. I am far more concerned that from a user perspective it looks like the instance was 'deleted' back to the point before I ran setup, and anything I deployed to OpenShift is gone.

gbraad commented 1 year ago

@cfergeau Any thoughts? This is not an extension related issue, but rather vfkit.

praveenkumar commented 1 year ago

Does crc status show instance is stopped or deleted?

deboer-tim commented 1 year ago

Deleted - see above:

CRC status

DEBU CRC version: 2.17.0+44e15711                 
DEBU OpenShift version: 4.12.9                    
DEBU Podman version: 4.4.1                        
DEBU Running 'crc status'                         
DEBU Running 'sw_vers -productVersion'            
crc does not seem to be setup correctly, have you run 'crc setup'?
praveenkumar commented 1 year ago

@deboer-tim did you face this still, or this happened with old version of all components?

cfergeau commented 1 year ago

crc does not seem to be setup correctly, have you run 'crc setup'? indicates an issue with the daemon:

pkg/crc/errors/multierror.go:const DaemonNotRunning daemonNotRunning = "crc does not seem to be setup correctly, have you run 'crc setup'?"

I guess when this happens ps aux |grep crc does not show crc processes. I'd expect crc setup && crc start to help recover from this though. I'm not sure how a crc delete or equivalent could happen without explicit user action.