search

ezmsg-org / ezmsg

Pure-Python DAG-based high-performance SHM-backed pub-sub and multi-processing pattern
https://ezmsg.readthedocs.io/en/latest/
MIT License
10 stars 5 forks source link

Request: Tests for graceful shutdown and force termination on KeyboardInterrupt #13

Open griffinmilsap opened 1 year ago

griffinmilsap commented 1 year ago

This behavior has been broken on Windows for at least one minor version, and is now fixed in 3.1.. I do most my dev on MacOS, but Windows is a significantly different platform and doesn't handle signals (and a lot of things) quite the same way as *nix systems.

I'd like a test that launches an ezmsg system that's supposed to run for two seconds, publishing some preset number of messages (20 @ 10 msgs/sec?) to a receiver that logs the messages to a file using MessageLogger before terminating with NormalTermination. The test should then interrupt the system with a KeyboardInterrupt (how???) after one second, and verifies that shutdown has happened gracefully before all expected messages were received/logged by the receiver. The system should shutdown without raising any exceptions to pass the test. It's important to write the test so that we cover "single process mode" where only one process will be executed (without multiprocessing) and a system where the publisher and receiver are both executed in different processes (because interrupts and graceful shutdown/termination are handled differently in this mode of execution).

Finally, a separate system should be written that tests the "force terminate" functionality that occurs when KeyboardInterrupt is sent again after graceful shutdown has occurred. This could be tested by blocking keyboard interrupt with the signal module by using the following line in the initialize function of a ez.Unit: signal.signal(signal.SIGINT, signal.SIG_IGN). The unit should have a single task that calls a blocking time.sleep for ~10 seconds. Again, after a second the test harness should inject a KeyboardInterrupt, and after another second, the test harness should inject a second Keyboard interrupt. Its okay for the system to raise exceptions when terminated this way; and the test passes if the system shuts down before 10 seconds elapses (ideally around 1-2 seconds of execution). This should be done with one unit (and one process) as well as multiple units/multi-process mode.

griffinmilsap commented 1 year ago

Also all tests should be implemented across fork and spawn multiprocessing!