quickemu-project / quicktest

Quickly and automatically test systems inside Quickemu virtual machines 🧑‍🔬
MIT License
16 stars 3 forks source link

bug: Failing test leaves VM running #27

Closed ali1234 closed 1 month ago

ali1234 commented 1 month ago

Expected behavior

Test runs should leave the host system in the same state they found it, regardless of whether they pass or fail.

Actual behavior

Failing tests leave the VM running. Closing the window causes it to keep running in the background, with no obvious way to shut it down. Subsequent runs of quicktest instantly fail because "the VM is already running".

Steps to reproduce the behavior

Run a test that fails. For example, change an OCR string so that it won't match, and then wait for it to give up.

Additional context

Quicktest output

Quicktest output ```text [20240518-013719] ⚠️ VM is already running. Stopping test case [20240518-013719] 🚨 Test failed. [20240518-013719] Last error in: VM is already running ```

Commentary

This is an annoyance when attempting to develop a testcase. If you spot that the test is going to fail and press ctrl-c then it will kill the VM. But if you don't notice until the test has failed and quicktest has exited, then there is no easy way to shut down the attached VM.

Quicktest should always clean up any VM it created when exiting. If interaction at the end of the test is desired, this should be implemented by making quicktest wait for user input. This should be an optional CLI switch, off by default.

popey commented 1 month ago

Yes, I was waiting for @flexiondotorg to add the --kill option, which is in master, but not landed in a release yet. While I could use various ways to kill the machine/process, I wanted to use something supported by quickemu. It's a bit messy to kill the process and delete the pid file etc.

Once Quickemu 4.9.5 (or newer) is released, I'll add it.