search

akelleh / causality

Tools for causal analysis
MIT License
1.06k stars 128 forks source link

Non-fully parallel jobs for causal estimation with large data sets. #80

Closed dcompgriff closed 2 years ago

dcompgriff commented 4 years ago

Yo. I realized that for large data sets your code sets the n_jobs field to 4 instead of -1. I'm guessing this is a typo since it should ideally run with as many cores as possible (i'm running 96 cores simultaneously).

It's in the 'nonparametric.py' script herehttps://github.com/akelleh/causality/blob/79cb779069b67f0d336745fc0d1aad343a3385fa/causality/estimation/nonparametric.py, line 34.

'self.defaults=EstimatorSettings(n_jobs=4, efficient=True)' should be 'self.defaults=EstimatorSettings(n_jobs=-1, efficient=True)'

akelleh commented 2 years ago

increased the default parallelism on master -- should land on pypi shortly