nrpe-ng can't regulary stop under Debian 10 (bootc/nrpe-ng#3)

[russellvt]

From bootc/nrpe-ng#3.

Feb 27 09:03:08 hostname systemd[1]: Started The next generation Nagios Remote Plugin Executor.
Feb 27 09:03:13 hostname systemd[1]: Stopping The next generation Nagios Remote Plugin Executor...
Feb 27 09:03:13 hostname nrpe-ng[1761987]: received SIGTERM, shutting down...
Feb 27 09:03:13 hostname nrpe-ng[1761987]: Exception in callback functools.partial(<function wrap.<locals>.null_wrapper at 0x7fb3e4b23f28>, <Future finished exception=AttributeError("'AsyncIOMainLoop' object has no attribute '_callb>
                                                 Traceback (most recent call last):
                                                   File "/usr/lib/python3/dist-packages/tornado/ioloop.py", line 758, in _run_callback
                                                     ret = callback()
                                                   File "/usr/lib/python3/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
                                                     return fn(*args, **kwargs)
                                                   File "/usr/lib/python3/dist-packages/tornado/ioloop.py", line 779, in _discard_future_result
                                                     future.result()
                                                   File "/usr/lib/python3/dist-packages/tornado/gen.py", line 326, in wrapper
                                                     yielded = next(result)
                                                   File "/usr/lib/python3/dist-packages/nrpe_ng/server/server.py", line 182, in sigterm_callback
                                                     if not io_loop.callbacks and not io_loop._timeouts:
                                                 AttributeError: 'AsyncIOMainLoop' object has no attribute '_callbacks'
Feb 27 09:04:43 hostname systemd[1]: nrpe-ng.service: State 'stop-sigterm' timed out. Killing.
Feb 27 09:04:43 hostname systemd[1]: nrpe-ng.service: Killing process 1761987 (nrpe-ng) with signal SIGKILL.
Feb 27 09:04:43 hostname systemd[1]: nrpe-ng.service: Main process exited, code=killed, status=9/KILL
Feb 27 09:04:43 hostname systemd[1]: nrpe-ng.service: Failed with result 'timeout'.
Feb 27 09:04:43 hostname systemd[1]: Stopped The next generation Nagios Remote Plugin Executor.

[russellvt]

Hey @TeTeHacko ... can you verify the version of nrpe-ng under Debian 10? I assume you installed it rom the distribution repo? Also, @infraweavers ... if you can help supply additional info, here?

[infraweavers]

@russellvt Been a while since we've used nrpe-ng; I don't think we were doing anything particularly crazy. If memory serves, we dropped in our standard list (check_cpu, check_load, check_mem) then restarted the service to add some more which is when we noticed how long it takes to restart.

[polski-g]

Same problem on Debian 11.

Can temporarily fix by modifying the unit file: KillSignal=SIGKILL

Relevant files are below:

root@localhost # /opt/nrpe-ng/venv/bin/python --version
Python 3.9.2

root@localhost # /opt/nrpe-ng/venv/bin/pip freeze
certifi==2022.6.15
charset-normalizer==2.1.0
docutils==0.19
idna==3.3
lockfile==0.12.2
nrpe-ng===0.2.1-rvt
python-daemon==2.3.0
requests==2.28.1
tornado==6.2
urllib3==1.26.10

root@localhost # cat /etc/systemd/system/nrpe-ng.service 
[Unit]
Description=NRPE-NG monitoring daemon
After=network.target

[Service]
Type=forking
PIDFile=/run/nrpe-ng/nrpe-ng.pid
ExecStart=/opt/nrpe-ng/venv/bin/nrpe-ng -c /etc/nrpe-ng/nrpe-ng.cfg -d

[Install]
WantedBy=multi-user.target
root@localhost # 

[TeTeHacko]

Updated to Debian 11, same version, same problem, same workaround working fine

nrpe-ng:
  Installed: 0.2.0-1
  Candidate: 0.2.0-1
  Version table:
 *** 0.2.0-1 500
        500 http://ftp.cz.debian.org/debian bullseye/main amd64 Packages
        100 /var/lib/dpkg/status

[russellvt]

Thank you for the update. I will need to get some VMs setup to test this one.