search

alteryx / featuretools

An open source python library for automated feature engineering
https://www.featuretools.com
BSD 3-Clause "New" or "Revised" License
7.25k stars 879 forks source link

Multiple threading version to calculate feature matrix #101

Closed dfayzur closed 6 years ago

dfayzur commented 6 years ago

Hello,

calculating feature matrix takes reasonable amount of time on single thread, while most of the other threads are doing nothing.

Is it possible to calculate feature matrix in parallel mode with multi threading support?

bschreck commented 6 years ago

Hi @dfayzur, we are currently working on a multicore version in PR #84. It should be merged in in the next few weeks, hopefully sooner. In the meantime, users have had success using Dask bags to parallelize calculate_feature_matrix. See our conversation with @jeff-hernandez on the Featuretools Gitter.

dfayzur commented 6 years ago

Thanks @bschreck. I will look to the Gitter.

Seth-Rothschild commented 6 years ago

Closing via #173