search

hengzhe-zhang / EvolutionaryForest

An open source python library for automated feature engineering based on Genetic Programming
GNU Lesser General Public License v3.0
134 stars 23 forks source link

Multiprocessing #73

Closed daizhonghao closed 2 years ago

daizhonghao commented 2 years ago

Description

Hi, there. Is parallel supported in this package. I notice there is a param "n_process", but when I use a dataset with #50_0000 times 100 featues, the program stucks there and I dont what is going on. 企业微信截图_16442195086386

hengzhe-zhang commented 2 years ago

Undoubtedly, parallel computing is supported by this package, and it is recommended to set the number of processes equal to the number of your computer processors. Moreover, to ascertain the reason for this fault, I suggest you run this package on a smaller dataset in scikit-learn to see whether parallel computing is supported by your computer.

daizhonghao commented 2 years ago

Undoubtedly, parallel computing is supported by this package, and it is recommended to set the number of processes equal to the number of your computer processors. Moreover, to ascertain the reason for this fault, I suggest you run this package on a smaller dataset in scikit-learn to see whether parallel computing is supported by your computer.

Hi, thanks for your quick reply. I tried a small dataset and it works. The parallel computing is supported on my computer. Do you have any requirements for the lightgbm version?

hengzhe-zhang commented 2 years ago

No, our package does not use LightGBM now. If you have a large dataset, use the tournament selection operator, e.g. "Tournament-7", instead of the lexicase selection operator might be a better choice. The speed of lexicase selection operator will be significantly improved in the next version of this package.

daizhonghao commented 2 years ago

No, our package does not use LightGBM now. If you have a large dataset, use the tournament selection operator, e.g. "Tournament-7", instead of the lexicase selection operator might be a better choice. The speed of lexicase selection operator will be significantly improved in the next version of this package.

Hi, thanks for your suggestion. I will try this setup. Again, thanks for your providing this useful tool.