Open arik-shurygin opened 16 hours ago
CSV idea is pretty neat 🤔 although it's seldom that we would run differential scenarios for each state at the moment... I think there's a good chance I would use it to be sure the book-keeping is right but not frequently...
Copied from another chat:
or you thinking of maximum flexibility: if for whatever reason run_task.py has three arguments, I can then flexibly csv of three columns to control this?
This is good and a bit more future proofing yeah
edit: breaking this into two separate issues, --explicit
flag will now be tracked here: https://github.com/CDCgov/scenarios-hpc-azure/issues/9
add a
--scenarios
flag which takes a list of string scenario names and to allow users to specify multiple scenarios per state. Thelaunch_experiment
script should then be able to pass along each scenario to therun_task.py
runner script via the --scenario flag on the runner.This will expand the
launch_experiment's
functionality and make it less likely that users need to write their own custom versions of the script.(edit: the following feature is now located in #9 ) Can also introduce an
explicit
mode which instead reads in a CSV that defines each task and the keywords passed along to it. For example, if yourrun_task.py
script takes in 2 arguments--state
,--scenario
, and the obligatory--jobid
you could provide a CSV with the two changing columns columnsstate, scenario
and then each row of that CSV could be launched as its own task. Since the user already provides the--jobid
flag to the launcher a similar row is not needed for that argument. I am curious if you would use such a mode @kokbent