The Rlearner interface uses cross_val_predict to predict mu but did not expose n_jobs params. The default value of n_jobs uses all processors, which may consumes too much memory (to OOM) especially for large dataset. The cross_val_predict method uses joblib for parallelization, the memory will grow with the number of workers. It's more reasonable to expose this param and let users decide.
Types of changes
What types of changes does your code introduce to CausalML?
Put an x in the boxes that apply
[ ] Bugfix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices apply)
Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.
Proposed changes
The Rlearner interface uses
cross_val_predictto predictmubut did not expose n_jobs params. The default value ofn_jobsuses all processors, which may consumes too much memory (to OOM) especially for large dataset. Thecross_val_predictmethod usesjoblibfor parallelization, the memory will grow with the number of workers. It's more reasonable to expose this param and let users decide.Types of changes
What types of changes does your code introduce to CausalML? Put an
xin the boxes that applyChecklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.Further comments
Nope