search

cmstas / HggAnalysisDev

3 stars 7 forks source link

Preselection: Implement Dask/condor submission for looper #1

Open sam-may opened 3 years ago

sam-may commented 3 years ago

Implement functionality to submit jobs to Dask and/or condor for the preselection looper.

Implementation would go in this function in prep_helper.py: https://github.com/cmstas/HggAnalysisDev/blob/a96044cf9bc66267fda5358d7d2e573fce32624d/Preselection/helpers/loop_helper.py#L151

Likely the cleanest way to do it would be to make a Batch directory and build helper classes for Dask/condor submission there.

Merging of output dataframes would also need to be updated accordingly: https://github.com/cmstas/HggAnalysisDev/blob/a96044cf9bc66267fda5358d7d2e573fce32624d/Preselection/helpers/loop_helper.py#L197-L208

Another thing to keep in mind: it would be nice to have the batch submission tools not be entirely specific to the looper (or at least easily generalizable), as they will also be useful for MVA training (e.g. hyperparameter scans) and Signal Region Optimization (scanning MVA cut values).

bsathian commented 3 years ago

I can take this up if no one else wants to