omnilib / aiomultiprocess

Take a modern Python codebase to the next level of performance.
https://aiomultiprocess.omnilib.dev
MIT License
1.77k stars 101 forks source link

Using multiprocessing.Value silently terminates the Process #179

Open rth opened 1 year ago

rth commented 1 year ago

If I try to use multiprocessing.Value for synchronization with aiomultiprocess.Process and uvloop even loop, the process gets silently terminated with no error messages.

Example

from aiomultiprocess import Process
import uvloop
from time import sleep
from multiprocessing import Value

async def func(signal):
    signal.value
    raise ValueError

if __name__ == '__main__':
    signal = Value('i', 0)
    proc = Process(                               
        target=func,                    
        loop_initializer=uvloop.new_event_loop,   
        args=(signal,),                      
    )                                             
    proc.start()                                  
    sleep(1)                                      
    assert proc.is_alive()

Expected behavior

In this example, I would expect the ValueError to be raised and displayed. This is indeed what happens if the signal.value line is removed.

Current behavior

Currently an assertion error is raised,

    assert proc.is_alive()
AssertionError

which means that the process is either never started or is terminated before reaching the line that raises the ValueError.

Am I doing something wrong?

Versions

Python 3.10.8 on Ubuntu Linux
aiomultiprocess         0.9.0
uvloop                  0.17.0
rth commented 1 year ago

FWIW, similar question when using a multiprocessing.Queue for synchronization with the created process. Submitted messages to the Queue inside the process are never received outside.

So I ended up using just plain multiprocessing with a basic implementation,

from multiprocessing import Process
import uvloop

def _run_in_loop(func, args):
    loop = uvloop.new_event_loop()
    loop.run_until_complete(func(*args))

def run_coroutine_in_process(func, *args) -> Process:
    process = Process(target=_run_in_loop, args=(func, args))
    process.start()
    return process

which appears to work with both multiprocessing.Value and multiprocessing.Queue with Python 3.11 (on Linux)