celery / billiard

Multiprocessing Pool Extensions
Other
405 stars 248 forks source link

Any task, even trivial, takes a lot of time using billiard #321

Open gianderiu opened 3 years ago

gianderiu commented 3 years ago

Any task, even trivial, takes a lot of time using billiard. The same test works fine using the multiprocessing library. test.py

import billiard as mp

def func(a):
  return a

y= list(range(10))

pool = mp.Pool(2)
s = sum(pool.map(func, y))
pool.close()
pool.join()

print (s)

run the script

time python3 test.py 
45

real    0m31.168s
user    0m0.104s
sys 0m0.023s

if you replace billiard with multiprocessing it works fine:

time python3 test.py 
45

real    0m0.185s
user    0m0.087s
sys 0m0.007s
gianderiu commented 3 years ago

The problem appears in release 3.6.1.0