MAIF / melusine

📧 Melusine: Use python to automatize your email processing workflow
https://maif.github.io/melusine
Other
352 stars 58 forks source link

use joblib for multiprocessing #1

Closed remiadon closed 5 years ago

remiadon commented 5 years ago

The current TransformerScheduler class uses a multiprocess version of pandas apply, imported from melusine.utils.multiprocessing

Actually scikit-learn already provides a high level interface for multiprocess/multithread processing over numpy/pandas, accessible from the external namespace

from sklearn.externals import joblib 

using joblib would reduce the code base and improve compatibility with the sklearn ecosystem