search

sgoldenlab / simba

SimBA (Simple Behavioral Analysis), a pipeline and GUI for developing supervised behavioral classifiers
https://simba-uw-tf-dev.readthedocs.io/
GNU General Public License v3.0
272 stars 137 forks source link

SHAP Multiprocessing Failed #360

Closed AndyWeasley2004 closed 1 month ago

AndyWeasley2004 commented 1 month ago

My error looks like this and I have set multicore to be True. IMG_0945

My CPU info is below: IMG_0946 Is multiprocessing not supported on this CPU? Thank you for your information.

sronilsson commented 1 month ago

Yes it should be supported, it should run them in parallel on each of your selected cores, your computer looks good.

I haven't seen this error before. It looks like it is trying to process some large value which is larger than or smaller than 2148483648 which is more than a 32bit floating point can handle... possibly? I am away for the next couple of days but will see if I can recreate this and get back to you.

sronilsson commented 1 month ago

... @AndyWeasley2004 One thing you could help me with:

If you try it on fewer cores - e.g., 5 does it run?

Or does it run of you build the model with fewer trees/estimators? Say 200?

It could be a RAM related issue if those things fixes it.

AndyWeasley2004 commented 1 month ago

I will try that when I go back to work. Thank you for your information!

sronilsson commented 1 month ago

👍🏻 I googled around and yes, the issue is discussed here as a possible memory error.

I don't know how much data you have, but if above doesn't fix it, check if it would run with less data (fewer, or smaller files inside the project_folder/csv/targets_inserted directory). If it runs then it would also suggest memory related.