Open john-waczak opened 1 year ago
This discussion looks relevant for setting up a slurm job array with a bunch of papermill tasks
This link provides a nice tutorial for configuring slurm job arrays. The easiest thing to do will probably be to have a script we can run that will generate generate the papermill parameter files as well as a slurm config file. That config file can contain the path of the parameter file as well as the output file path we want to use.
Once we have demo notebooks for each analysis style, we should augment them with a set of parameters that can be passed in at execution time via papermill. For starters these should include:
start_datetime
end_datetime
device_name
We should also update the quarto metadata to use the supplied
device_name
parameter when generating the title for each report. Thedevice_name
parameter should also be injected into the output notebook filename to keep things easy.