python / cpython

The Python programming language
https://www.python.org
Other
63.34k stars 30.32k forks source link

Multiprocessing does not work properly when using the trace module. #82692

Open 47f9b97d-caf3-494f-b616-d3fb0d9bbfaa opened 5 years ago

47f9b97d-caf3-494f-b616-d3fb0d9bbfaa commented 5 years ago
BPO 38511
Nosy @pfmoore, @tjguk, @zware, @zooba, @rls1004
Files
  • test.py: test code
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', '3.9', 'OS-windows'] title = 'Multiprocessing does not work properly when using the trace module.' updated_at = user = 'https://github.com/rls1004' ``` bugs.python.org fields: ```python activity = actor = '\xd8\xaa\xd8\xa7\xd8\xb4\xd9\x8a\xd8\xb1\xd8\xa7\xd8\xaa \xd9\x85\xd8\xb3\xd8\xa7\xd9\x86\xd8\xaf' assignee = 'none' closed = False closed_date = None closer = None components = ['Windows'] creation = creator = 'rls1004' dependencies = [] files = ['48667'] hgrepos = [] issue_num = 38511 keywords = [] message_count = 2.0 messages = ['354864', '354879'] nosy_count = 6.0 nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'steve.dower', 'rls1004', '\xd8\xaa\xd8\xa7\xd8\xb4\xd9\x8a\xd8\xb1\xd8\xa7\xd8\xaa \xd9\x85\xd8\xb3\xd8\xa7\xd9\x86\xd8\xaf'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue38511' versions = ['Python 3.9'] ```

    47f9b97d-caf3-494f-b616-d3fb0d9bbfaa commented 5 years ago

    normal result : $ python test.py {'p1': 1, 'p2': 1, 'p3': 1, 'p4': 1}

    run with tracing : $ python -mtrace --trackcalls test.py {}

    It seems that the foo and save functions that multiprocess should call are not called.

    ghost commented 5 years ago
    #!/usr/bin/env python
    import multiprocessing, sys, trace
    from functools import partial
    
    num_list = ['p1', 'p2', 'p3', 'p4']
    
    def foo(name):
        print(name+'\n')
        return name
    def save(result, shared):
        print('a\n')
        shared.results[result] = 1
    
    def mm():
        manager = multiprocessing.Manager()
        shared = manager.dict()
        shared.results = dict()
    
        clbk = partial(save, shared=shared)
    
        pool = multiprocessing.Pool(processes=2)
        serial_pool = multiprocessing.Pool(1)
    
        for name in num_list:
            serial_pool.apply_async(foo, args=[name], callback=clbk)
    
        pool.close()
        pool.join()
    
        print(shared.results)
    
    mm()