python / cpython

The Python programming language
https://www.python.org
Other
63.73k stars 30.53k forks source link

Cannot pass a SyncManager proxy to a multiprocessing subprocess on Windows #77269

Open d15633c2-b981-4c58-ad1d-90d0637c71ab opened 6 years ago

d15633c2-b981-4c58-ad1d-90d0637c71ab commented 6 years ago
BPO 33088
Nosy @pfmoore, @pitrou, @tjguk, @zware, @zooba, @applio

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', 'OS-windows'] title = 'Cannot pass a SyncManager proxy to a multiprocessing subprocess on Windows' updated_at = user = 'https://bugs.python.org/jjdmon' ``` bugs.python.org fields: ```python activity = actor = 'ned.deily' assignee = 'none' closed = False closed_date = None closer = None components = ['Windows'] creation = creator = 'jjdmon' dependencies = [] files = [] hgrepos = [] issue_num = 33088 keywords = [] message_count = 1.0 messages = ['313964'] nosy_count = 7.0 nosy_names = ['paul.moore', 'pitrou', 'tim.golden', 'zach.ware', 'steve.dower', 'davin', 'jjdmon'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue33088' versions = ['Python 3.6'] ```

d15633c2-b981-4c58-ad1d-90d0637c71ab commented 6 years ago

The following simple example code creates a started SyncManager and passes it as an argument to a subprocess started with multiprocessing.Process(). It works on Linux and Mac OS but fails on Windows.

import multiprocessing as mp

def subProcFn(m1):
    pass

if __name__ == "__main__":

    __spec__ = None

    m1 = mp.Manager()

    p1 = mp.Process(target=subProcFn, args=(m1,))
    p1.start()
    p1.join()

This is the traceback in Spyder:

runfile('D:/ManagerBug.py', wdir='D:')
Traceback (most recent call last):

File "\<ipython-input-1-534b9087bae9>", line 1, in \<module> runfile('D:/ManagerBug.py', wdir='D:')

File "...\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile execfile(filename, namespace)

File "...\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/ManagerBug.py", line 22, in \<module> p1.start()

File "...\anaconda3\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self)

File "...\anaconda3\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj)

File "...\anaconda3\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj)

File "...\anaconda3\lib\multiprocessing\popen_spawnwin32.py", line 65, in \_init__ reduction.dump(process_obj, to_child)

File "...\anaconda3\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj)

TypeError: can't pickle weakref objects