[All] Use of atexit in tests/__init__.py conflicts with walk_packages()

[qris]

Summary

• OS: All
• Architecture: All
• Psutil version: 5.4.6 to 5.8.0 and current HEAD
• Python version: 3.7.3 (not that it matters)
• Type: tests

Description

walk_packages() imports all packages that it can find. From its docstring:

Note that this function must import all *packages* (NOT all
modules!) on the given path, in order to access the __path__
attribute to find submodules.

Unfortunately if psutil is installed then it will walk into psutil.tests, which imports __init__.py, which has side-effects on import:

atexit.register(DEVNULL.close)
...
@atexit.register
def cleanup_test_procs():
    reap_children(recursive=True)

# atexit module does not execute exit functions in case of SIGTERM, which
# gets sent to test subprocesses, which is a problem if they import this
# module. With this it will. See:
# https://gmpy.dev/blog/2016/how-to-always-execute-exit-functions-in-python
if POSIX:
    signal.signal(signal.SIGTERM, lambda sig, frame: sys.exit(sig))

Thus it will attempt to kill any children of your current process after doing so. Note that we don't actually have to run any psutil tests to trigger this.

For example I have a process that collects modules and their exports (for auto-complete in an IDE) and when it terminates it outputs:

2021-10-25 14:56:25.335:79933:warnings:110:WARNING: .../lib/python3.7/site-packages/psutil-5.4.6-py3.7-linux-x86_64.egg/psutil/tests/__init__.py:1156: UserWarning:couldn't terminate process psutil.Process(pid=79969, name='python3', started='14:55:15'); attempting kill()

I think we should only register atexit hooks in a process/script, not in a library (or part of one, such as a package's __init__.py).

[giampaolo]

walk_packages() imports all packages that it can find

What is walk_packages?

[qris]

Sorry, pkgutil.walk_packages(), part of the Python standard library.