python / cpython

The Python programming language
https://www.python.org
Other
63.21k stars 30.27k forks source link

multiprocessing.dummy missing cpu_count #74054

Open 5d07e2db-5903-4394-a466-2a62be49b70c opened 7 years ago

5d07e2db-5903-4394-a466-2a62be49b70c commented 7 years ago
BPO 29868
Nosy @vstinner, @florentx, @applio, @wiseman

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', 'library'] title = 'multiprocessing.dummy missing cpu_count' updated_at = user = 'https://github.com/wiseman' ``` bugs.python.org fields: ```python activity = actor = 'vstinner' assignee = 'none' closed = False closed_date = None closer = None components = ['Library (Lib)'] creation = creator = 'johnwiseman' dependencies = [] files = [] hgrepos = [] issue_num = 29868 keywords = [] message_count = 2.0 messages = ['289950', '297640'] nosy_count = 4.0 nosy_names = ['vstinner', 'flox', 'davin', 'johnwiseman'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue29868' versions = ['Python 3.6'] ```

Linked PRs

5d07e2db-5903-4394-a466-2a62be49b70c commented 7 years ago

The documentation for the multiprocessing.dummy module says that it "replicates the API of multiprocessing." In Python 2.7, I can import multiprocessing.dummy and use the cpu_count function:

$ python2
Python 2.7.12 (default, Oct 29 2016, 19:21:06)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing.dummy as mp
>>> mp.cpu_count()
8

But in Python 3.6, multiprocessing.dummy is missing cpu_count:

$ python3
Python 3.6.0 (default, Mar 21 2017, 13:27:21)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing.dummy as mp
>>> mp.cpu_count()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'multiprocessing.dummy' has no attribute 'cpu_count'

I would expect that multiprocessing.dummy would have cpu_count, since that function is available in multiprocessing, and it's there in Python 2.7.

It looks like it was removed in commit 04842a8, "Remove unused or redundant imports in concurrent.futures and multiprocessing" (Florent Xicluna 5 years ago). Maybe the removal was inadvertent?

I realize there are several workarounds, but based on the documentation and the behavior of previous versions, I wouldn't have expected this breaking change.

vstinner commented 7 years ago

multiprocessing.dummy lacks many symbols:

>>> set(multiprocessing.__all__) - set(multiprocessing.dummy.__all__)
{'RawArray', 'set_forkserver_preload', 'ProcessError', 'RawValue', 'BufferTooShort', 'reducer', 'set_executable', 'Array', 'cpu_count', 'get_all_start_methods', 'get_start_method', 'AuthenticationError', 'Value', 'set_start_method', 'allow_connection_pickling', 'get_logger', 'SimpleQueue', 'TimeoutError', 'log_to_stderr', 'get_context'}