_Process instances leaked

Problem

_Process instances created via the simulator.process() method are never released and therefore they (and their contained greenlet instance are leaked). This can be a big issue for long running simulations that dynamically create processes. This is also means that any local variables in the process methods will also be leaked.

Reproducer

from simulus import simulator

sim = simulator()

def thread():
    sim.sleep(5)

procs = [sim.process(thread) for _ in range(0, 10)]

# Run all processes to completion
sim.run()
assert sim.now == 5

# Printing the greenlet will indicate "... suspended active started"
print(procs[0].vert)

# Now lets look for memory leaks...
import gc
del procs
gc.collect()
objs = gc.get_objects()
# Grab the first leaked _Process object and see who is holding onto it...
p = next(filter(lambda t: t.__class__.__name__ == "_Process", objs))
refs = gc.get_referrers(p)

# One of the referrers will be: <bound method _Process.invoke> (this is the method that is passed to the greenlet constructor)

Based on the above, I believe that the greenlet associated with the _Process is never fully being cleaned up and therefore its reference to the owning _Process (via the invoke method) is never being released and therefore the garbage collector cannot free the _Process.

Cause

After examining the _Process code, it appears that the problem is the _Process.terminate() method is never allowed to return. Instead, it explicitly calls self.main.switch. This means that the contained greenlet's reference to the _Process.invoke method will never complete (because the code always switches to another greenlet right before it completes) and therefore the greenlet infrastructure doesn't know that it can cleanup the thread because technically it never finishes.

Proposed Solution:

In _Process.terminate change:

self.main.switch()

to:

self.vert.parent = self.main

Greenlet will automatically switch to a dead thread's parent ("When a greenlet dies, in which case execution jumps to the parent greenlet." ref: https://greenlet.readthedocs.io/en/latest/switching.html).

Proof

I made the above change locally and reran my reproducer. Now when I examine the vert instance of a completed process it is reported as "dead". When I run the garbage collection steps in the reproducer, no instances of _Process are found.

liuxfiu / simulus