Open 50bdd440-9dbe-4e66-8581-0e32a067841b opened 5 years ago
Add a human friendly names to the threads opened by multiprocessing.pool.Pool and multiprocessing.pool.ThreadPool objects.
Sample usage: ThreadPool(name="ClientsPool", processes=8)
Thanks for the report. I think using current_thread().name gives thread name with number that is useful. I don't know the exact use case or maybe I am misunderstanding the use case and perhaps adding a script where your PR applies with the output will be helpful. I will resort to feedback from others. Adding Antoine as per the expert index for multiprocessing module. Antoine, feel free to remove yourself if this is irrelevant.
# bpo-34720.py
from multiprocessing.pool import ThreadPool
def f(x):
from threading import current_thread
print(current_thread().name)
return x*x
if __name__ == '__main__':
with ThreadPool(5) as p:
print(p.map(f, [1, 2, 3]))
$ ./python.exe ../backups/bpo34720.py
Thread-1
Thread-1
Thread-2
[1, 4, 9]
# With PR and name as "custom-name" for ThreadPool
from multiprocessing.pool import ThreadPool
def f(x):
from threading import current_thread
print(current_thread().name)
return x*x
if __name__ == '__main__':
with ThreadPool(5, name="custom-name") as p:
print(p.map(f, [1, 2, 3]))
git:(pr_9906) ./python.exe ../backups/bpo34720.py custom-name-Worker-0 custom-name-Worker-1 custom-name-Worker-2 [1, 4, 9]
Hi Raz,
Thank you
The default name of the threads does not allow differentiation between pool threads and other threads. This problem is more notable when you have several thread pools. Also, since there are some management threads to the pool, and one might want to know which is which. It was very helpful for me while debugging some memory issues in my project, this is where the idea came from.
The names themselves were my idea, but I welcome any change to them.
Thanks for posting this. I think this is a good idea, will take a look at the PR later.
Thanks for clarifying the use case. I think it's a good idea with multiple thread pools and below outputs for the program with the PR is more easier to debug than the current code.
# ../backups/bpo34720.py
from multiprocessing.pool import ThreadPool
def f(x):
from threading import current_thread
print(current_thread().name)
return x*x
if __name__ == '__main__':
p1 = ThreadPool(5, name="process-1")
p1.map_async(f, [1, 2, 3])
p1.close()
p2 = ThreadPool(5, name="process-2")
p2.map_async(f, [1, 2, 3])
p2.close()
p1.join()
p2.join()
# With patch
$ ./python.exe ../backups/bpo34720.py
process-1-Worker-0
process-1-Worker-1
process-1-Worker-0
process-2-Worker-0
process-2-Worker-1
process-2-Worker-0
# Without patch and removing name parameter
$ python3.7 ../backups/bpo34720.py
Thread-1
Thread-1
Thread-1
Thread-9
Thread-9
Thread-9
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields: ```python assignee = None closed_at = None created_at =
labels = ['3.8', 'type-feature', 'library']
title = 'Add name to process and thread pool'
updated_at =
user = 'https://github.com/RazManorAllegro'
```
bugs.python.org fields:
```python
activity =
actor = 'xtreak'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation =
creator = 'Raz Manor'
dependencies = []
files = []
hgrepos = []
issue_num = 34996
keywords = ['patch']
message_count = 6.0
messages = ['327819', '327865', '327900', '328196', '328199', '328239']
nosy_count = 4.0
nosy_names = ['pitrou', 'matrixise', 'xtreak', 'Raz Manor']
pr_nums = ['9906']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue34996'
versions = ['Python 3.8']
```
Linked PRs