Use argparse to define the arguments passed to the scripts. This gives us automatic --help/-h arguments and makes the code easier to maintain.
Split read_params_from_cmdline() for job and main script.
read_params_from_cmdline() was previously used by both the cluster_utils main script (e.g. grid_search.py) and the job scripts. They have different requirements and thus the function was pretty convoluted. With this change now
read_params_from_cmdline() is only responsible only for the job scripts. It is refactored and uses argparse internally for hopefully better readable code and --help-support.
read_main_script_params_with_smart_settings() is the counter-part for the grid_search/hp_optimization scripts.
BREAKING 1:read_params_from_cmdline(): The job script now expects named arguments instead of positional ones. This makes the optionality of server information and distinction between settings file vs dictionary string much easier to implement.
This affects manual calls of the job script and may break non-python job scripts which operate on the arguments.
BREAKING 2: Support for custom hooks in read_params_from_cmdline() has been removed as it doesn't seem to be relevant here. I'm not sure, though, if this is okay or if some people depend on it in the job script. If in doubt, I'll add it back.
--help/-harguments and makes the code easier to maintain.read_params_from_cmdline()for job and main script.read_params_from_cmdline()was previously used by both the cluster_utils main script (e.g. grid_search.py) and the job scripts. They have different requirements and thus the function was pretty convoluted. With this change nowread_params_from_cmdline()is only responsible only for the job scripts. It is refactored and uses argparse internally for hopefully better readable code and--help-support.read_main_script_params_with_smart_settings()is the counter-part for the grid_search/hp_optimization scripts.BREAKING 1:
read_params_from_cmdline(): The job script now expects named arguments instead of positional ones. This makes the optionality of server information and distinction between settings file vs dictionary string much easier to implement. This affects manual calls of the job script and may break non-python job scripts which operate on the arguments.BREAKING 2: Support for custom hooks in
read_params_from_cmdline()has been removed as it doesn't seem to be relevant here. I'm not sure, though, if this is okay or if some people depend on it in the job script. If in doubt, I'll add it back.Closes #103
How I tested