CDCgov / scenarios-hpc-azure

Apache License 2.0
0 stars 0 forks source link

add `--scenarios` flag #8

Open arik-shurygin opened 16 hours ago

arik-shurygin commented 16 hours ago

add a --scenarios flag which takes a list of string scenario names and to allow users to specify multiple scenarios per state. The launch_experiment script should then be able to pass along each scenario to the run_task.py runner script via the --scenario flag on the runner.

This will expand the launch_experiment's functionality and make it less likely that users need to write their own custom versions of the script.

(edit: the following feature is now located in #9 ) Can also introduce an explicit mode which instead reads in a CSV that defines each task and the keywords passed along to it. For example, if your run_task.py script takes in 2 arguments --state, --scenario, and the obligatory --jobid you could provide a CSV with the two changing columns columns state, scenario and then each row of that CSV could be launched as its own task. Since the user already provides the --jobid flag to the launcher a similar row is not needed for that argument. I am curious if you would use such a mode @kokbent

kokbent commented 16 hours ago

CSV idea is pretty neat 🤔 although it's seldom that we would run differential scenarios for each state at the moment... I think there's a good chance I would use it to be sure the book-keeping is right but not frequently...

kokbent commented 16 hours ago

Copied from another chat:

or you thinking of maximum flexibility: if for whatever reason run_task.py has three arguments, I can then flexibly csv of three columns to control this?

This is good and a bit more future proofing yeah

arik-shurygin commented 14 hours ago

edit: breaking this into two separate issues, --explicit flag will now be tracked here: https://github.com/CDCgov/scenarios-hpc-azure/issues/9