[Linux] Process.children() hogs CPU

[TheTechromancer]

Summary

• OS: Arch Linux
• Architecture: amd64
• Psutil version: 5.9.5
• Python version: 3.11.6
• Type: performance

Description

Hi, first off thanks for making this library; it's very useful. Recently I've been running into an issue on Linux where calls to Process.children(recursive=True) take up more CPU than expected.

As you can see from the cProfile output, the call takes roughly 8 seconds of CPU time. This is strange because there are only a few subprocesses (five or less). The program is using asyncio, so there are quite a few active asyncio tasks; but these should all exist within the same thread.

It appears the real culprit here is _pslinux.py / ppid_map():

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   16.932   16.932 scanner.py:557(stop)
        2    0.000    0.000   16.837    8.418 misc.py:1071(kill_children)
        4    0.000    0.000   16.826    4.207 __init__.py:270(wrapper)
        2    0.000    0.000   16.800    8.400 __init__.py:881(children)
        2    0.011    0.006   16.769    8.385 _pslinux.py:1624(ppid_map)

Here is the python code that calls it:

def kill_children(parent_pid=None, sig=signal.SIGTERM):
    """
    Forgive me father for I have sinned
    """
    try:
        parent = psutil.Process(parent_pid)
    except psutil.NoSuchProcess:
        log.debug(f"No such PID: {parent_pid}")
    log.debug(f"Killing children of process ID {parent.pid}")
    children = parent.children(recursive=True) # <---------
    for child in children:
        log.debug(f"Killing child with PID {child.pid}")
        if child.name != "python":
            try:
                child.send_signal(sig)
            except psutil.NoSuchProcess:
                log.debug(f"No such PID: {child.pid}")
            except psutil.AccessDenied:
                log.debug(f"Error killing PID: {child.pid} - access denied")

[giampaolo]

The ppid_map() call takes indeed a long time, but with this report alone it's impossible to tell why. Ticket is 6 months old so I assume you can no longer reproduce this issue, correct? If that's the case then please close it.

[TheTechromancer]

Yes; sadly I wasn't able to reproduce this in isolation. Closing for now.