zeehio / parmap

Easy to use map and starmap python equivalents
Apache License 2.0
144 stars 9 forks source link

Cant' use pm_bar with pm_pool #21

Closed CarloNicolini closed 5 years ago

CarloNicolini commented 5 years ago

I have noticed that using both pm_bar and pm_pool hides the progress bar. Here a MWE:

import parmap
def myfunction(*args, **kwargs):
    return args[0]+args[1]
from functools import partial
# In this way the progress bar is shown
result = parmap.starmap(partial(myfunction, x='a'), list(itertools.product([3,4],[1,2])), pm_pbar=True)

# In this way it is not shown
from contextlib import closing 
from multiprocessing import Pool, cpu_count
with closing( Pool(cpu_count()//2) ) as pool:
    res = parmap.starmap(partial(myfunction, x='a'), list(itertools.product([3,4],[1,2])), pm_pbar=True, pm_pool=pool)

Any idea on the reason?

zeehio commented 5 years ago

On my first version of the progress bar I was relying on some private variables of the multiprocessing module and I wasn't sure they would behave properly when a custom pool was passed.

It seems it works as expected, so now it should be possible to use pm_pbar and pm_pool simultaneously.

Sorry for not replying before.