Possibility of supporting CTRL_BREAK_EVENT to stop processes

[lbianchi-lbl]

We've been using Supervisor to manage auxiliary processes for a cross-platform application including, thanks to this fork, Windows 10.

Recently, we introduced code to perform a clean shutdown when the application receives a signal to stop it (SIGINT, SIGTERM, in addition to the standard Python KeyboardInterrupt). This, in combination with Supervisor, works just fine on Linux and macOS, but on Windows the application logs show that the clean shutdown code is not executed after stopping or restarting the process with Supervisor.

After researching the issue (this SO answer in particular looks very thorough), my understanding is:

• Windows does not support sending arbitrary signals to other processes at the system level
• The only reliable way of ending a process is to kill it (which is what supervisor-win does), but this cannot be intercepted by the process and thus prevents the process from initiating a clean shutdown when the signal is received
• Sending a CTRL_BREAK_EVENT seems to be the only semi-reliable way to send a signal to another non-GUI process so that the signal can be caught by the application and trigger the clean shutdown
• CTRL_BREAK_EVENT seems to successfully trigger a clean shutdown, both in Python applications (where it is detected as the interceptable signal signal.SIGBREAK/21) and in non-Python applications (e.g. mongod, the MongoDB server)
• CTRL_BREAK_EVENTs are sent not only to the target process, but also to the entire process group that the process belongs to
• When a child process is created on Windows using supbrocess.Popen, the child process will belong to the same process group as the parent process, unless the creationflags=subprocess.CREATE_NEW_PROCESS_GROUP kwarg is provided

Experimentally, given a parent process A and a child process B, the following scenario seems to result in A being able to send a CTRL_BREAK_EVENT to B without A being terminated itself:

import subprocess
import signal

popen = subprocess.Popen(['B', '--some', '--args'], creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
popen.send_signal(signal.CTRL_BREAK_EVENT)

My question at this point is: is there a way to support this functionality in supervisor-win, so that managed processed can be ended with CTRL_BREAK_EVENT instead of being terminated?

From my tests, I was not able to find a way to do so using the documented configuration options. In particular, specifying stopsignal=1 (the number corresponding to signal.CTRL_BREAK_EVENT on Windows) results in the supervisord process itself being terminated.

This is consistent to what stated above: if this flag is not provided when creating the managed process (which seems to be the case with the supervisor-win versions I have tested, 4.1.1 and 4.3.0), this solution does not work because the supervisord process and the managed processes belong to the same process group, and so issuing a CTRL_BREAK_EVENT signal will cause both the managed process and the supervisord process to terminate.

Naively, it would seem that a way of achieving the correct behavior would be to add creationflags=subprocess.CREATE_NEW_PROCESS_GROUP (possibly exposing it as a configuration parameter) when calling subprocess.Popen in process.py; naturally, being unfamiliar with the codebase and with Windows systems programming in general, I'm well aware that there might be many reasons why this would not work, or would cause other issues, and that therefore the best thing to do would be ask you directly for clarifications.

My final questions are then:

• Is this solution something that has been considered or tried for supervisor-win?
• If not, would there be interest in implementing it?
• If yes, what are the known or possible roadblocks?

Thank you one more time for all your work on this very useful tool!

EDIT 2020-05-14 The interceptable signal corresponding to CTRL_BREAK_EVENT is signal.SIGBREAK, with a numeric value of 21 instead of 22 as previously written above

[alexsilva]

Based on what you explained I was able to add support for CTRL_ signals.

In the process settings you can set the following value

[comand:process]
stopsignal=CTRL_BREAK_EVENT

When executing the command: supervisor> stop process

The signal will be emitted CTRL_BREAK_EVENT

Another possibility is to send the signal directly to the process:

supervisor> signal CTRL_BREAK_EVENT process

To test the supervisor with these changes: python -m pip install git+https://github.com/alexsilva/supervisor.git@windows -U

[lbianchi-lbl]

Thank you for your quick response, this is awesome! I can confirm that, with the latest version at this time (commit b3ea562), everything works as expected after setting stopsignal=CTRL_BREAK_EVENT in the Supervisor configuration file.

For future reference, on the application side, the following seems to work as a cross-platform way to handle stop signals:

import logging, signal, sys

LOG = logging.getLogger(__name__)

def _handle_stop_signals(sig_num, frame):
    # customize behavior, e.g. raise exception
    LOG.info('Received stop signal %d', sig_num)

def register_stop_signals():
    stop_signals = [
        signal.SIGINT,
        signal.SIGTERM,
    ]
    if sys.platform == 'win32':
        stop_signals.append(signal.SIGBREAK)

    for sig in stop_signals:
        LOG.debug('Registering stop signal %s (code=%d)', signal.Signals(sig), sig)
        signal.signal(sig, _handle_stop_signals)

Thank you one more time for implementing this so quickly!

[alexsilva]

Thanks for the contribution.