Parallel R sessions are initialized and terminated each time crossval is called. This creates substantial unnecessary overhead in the computation of crosspred, where ensembles are computed on each split sample, and in the computation of ddml_iv, where multiple cross-sample predictions are computed.
A possible solution: consider adding anoter input to crossval, indicating whether clusteres should be newly initialized or whether existing clusteres should be utilized.
thomaswiemann
commented
1 year ago
Parallel R sessions are initialized and terminated each time
crossvalis called. This creates substantial unnecessary overhead in the computation ofcrosspred, where ensembles are computed on each split sample, and in the computation ofddml_iv, where multiple cross-sample predictions are computed.A possible solution: consider adding anoter input to
crossval, indicating whether clusteres should be newly initialized or whether existing clusteres should be utilized.