Improve CLI for Workflow

[sgoldenCS]

Currently, our drivers usually use argparse (or worse, nothing at all) to parse input arguments. This works ok, but requires explicit definitions for every configurable argument and can get quite messy. Even when set up well, using argparse to set up environments and configurations requires very long commands.

My suggestion is we build CLI tools using something like Click or Typer (python modules for CLI) that allow for loading an "environment" of sorts that can be modified with further commands. I think it would work well with the DAG runtime that I built a while ago.

An example I could see would be something like:

workflow load <workflow "environment">
workflow config set_file
workflow config view
workflow config set <key> <value>
workflow modules list     # lists modules used by current workflow
# or alternatively 
# workflow config update <file to pull updates from>
workflow run ifarm

Of course, you could run multiple configuration set up commands, the set or update commands could ask for confirmation of a change (e.g. "You are changing from to . Do you want to continue?") which could be avoided with a -y command (like pip installs). The run command could include extra add-ons like ifarm or polaris that would call special scripts to properly allocate GPUs/Nodes/etc on the respective machines.

These behaviors are somewhat inspired from using things like the git and docker CLI systems which use sub-commands to access just a portion of the system.

Unfortunately, I don't have any projects that I can do this work for at the moment, so it may end up languishing here for a while. Just wanted to make sure the idea was captured in writing somewhere.

[sgoldenCS]

Just wanted to add the websites for Click and Typer if people are curious. Click seems to be the faster, lighter one, but Typer has some nice autocomplete things that might be worth it for us.

Click: https://click.palletsprojects.com/en/8.1.x/ Typer: https://typer.tiangolo.com/