This makes some attempts to clean up during shutdown.
The main thing is the atexit function in the run_pipeline function which simply closes a run/node_run when shutting down.
The atexit in run() will shutdown any still running processes.
It also attempts to kill the run_process when the run_pipeline process is shut down via strg+c. This should trigger the atexit function in run() so at least the children are killed. This does not do any attempt to do the same for any other signal yet
Also included is a fix to actually check all ancestors of a task when checking if any of them is already failed -> the effect is that we do not schedule any tasks from an already queued sub pipelines (like a parallel task) in case the parent of that subpipeline is failed.
With this in place I could successfully add some signal handler (not included yet, needs some more testing) which kills the running processes and closes the runs. It also handles strg+c in flask better, at least I didn't see leftover processes anymore.
This makes some attempts to clean up during shutdown.
run_pipelinefunction which simply closes a run/node_run when shutting down.run()will shutdown any still running processes.run()so at least the children are killed. This does not do any attempt to do the same for any other signal yetAlso included is a fix to actually check all ancestors of a task when checking if any of them is already failed -> the effect is that we do not schedule any tasks from an already queued sub pipelines (like a parallel task) in case the parent of that subpipeline is failed.
With this in place I could successfully add some signal handler (not included yet, needs some more testing) which kills the running processes and closes the runs. It also handles strg+c in flask better, at least I didn't see leftover processes anymore.
partly covers: https://github.com/mara/data-integration/issues/40