csinva / imodels

Interpretable ML package 🔍 for concise, transparent, and accurate predictive modeling (sklearn-compatible).
https://csinva.io/imodels
MIT License
1.36k stars 121 forks source link

`FIGSRegressor.fit()` does not seem to utilize all CPU cores. #94

Open tigerinus opened 2 years ago

tigerinus commented 2 years ago

I have a training dataset of 1473711 records.

After throwing it to FIGSRegressor.fit(), it's been running for almost 3hrs without evidence of stopping.

Looking at the processes I see there are 4 running parallel:

259883 Sl     0:00 /usr/bin/python -c from joblib.externals.loky.backend.resource_tracker import main; main(60, False)
259885 S      1:47 /usr/bin/python -m joblib.externals.loky.backend.popen_loky_posix --process-name LokyProcess-1 --pipe 73
259886 S      2:18 /usr/bin/python -m joblib.externals.loky.backend.popen_loky_posix --process-name LokyProcess-2 --pipe 74
259887 S      2:01 /usr/bin/python -m joblib.externals.loky.backend.popen_loky_posix --process-name LokyProcess-3 --pipe 75
259888 S      1:48 /usr/bin/python -m joblib.externals.loky.backend.popen_loky_posix --process-name LokyProcess-4 --pipe 77

However in top I see only 1/4 CPU getting utilized:

image

Any chance this is a bug or something expected?

Thanks.