opensistemas-hub / osbrain

osBrain - A general-purpose multi-agent system module written in Python
https://osbrain.readthedocs.io/en/stable/
Apache License 2.0
175 stars 43 forks source link

Friendly parallelization API for simple tasks #300

Open Peque opened 6 years ago

Peque commented 6 years ago

Embrace multiprocessing.Pool or joblib API:

https://stackoverflow.com/questions/9786102/how-do-i-parallelize-a-simple-python-loop

Or concurrent.futures.ProcessPoolExecutor():

https://towardsdatascience.com/heres-how-you-can-get-a-2-6x-speed-up-on-your-data-pre-processing-with-python-847887e63be5

None of them provide advanced serialization capabilities.

See robinvol-backtest/pool.py.