python / cpython

The Python programming language
https://www.python.org
Other
63.13k stars 30.22k forks source link

Multiprocessing.pool API Extension - Pass Data to Workers w/o Globals #79018

Open 2052b960-6bf7-4425-9da1-66a97f12e1f8 opened 6 years ago

2052b960-6bf7-4425-9da1-66a97f12e1f8 commented 6 years ago
BPO 34837
Nosy @terryjreedy, @pitrou, @applio, @seanharr11
PRs
  • python/cpython#9627
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['3.8', 'type-feature', 'library'] title = 'Multiprocessing.pool API Extension - Pass Data to Workers w/o Globals' updated_at = user = 'https://github.com/seanharr11' ``` bugs.python.org fields: ```python activity = actor = 'terry.reedy' assignee = 'none' closed = False closed_date = None closer = None components = ['Library (Lib)'] creation = creator = 'seanharr11' dependencies = [] files = [] hgrepos = [] issue_num = 34837 keywords = ['patch'] message_count = 2.0 messages = ['327168', '327169'] nosy_count = 4.0 nosy_names = ['terry.reedy', 'pitrou', 'davin', 'seanharr11'] pr_nums = ['9627'] priority = 'normal' resolution = None stage = 'patch review' status = 'open' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue34837' versions = ['Python 3.8'] ```

    terryjreedy commented 6 years ago

    Quoting the PR:

    Proposing a new kwarg in the __init__() method of multiprocessing.Pool named expect_initret.

    This kwarg defaults to False. When set to True, the return value of the initializer function is passed to the function we are applying (i.e. func in Pool.map(func, ls)), as a kwarg named initret.

    This PR includes thorough test coverage, and provides backwards compatibility (at least in Python3.x).

    See blog post https://thelaziestprogrammer.com/python/multiprocessing-pool-expect-initret-proposal for example use cases, as well as an initial defense for why this makes the multiprocessing.Pool API more approachable, to more Python end-users, and augments the library.

    terryjreedy commented 6 years ago

    New features only go in next version.