Open sgoldenCS opened 1 month ago
Just wanted to add the websites for Click and Typer if people are curious. Click seems to be the faster, lighter one, but Typer has some nice autocomplete things that might be worth it for us.
Click: https://click.palletsprojects.com/en/8.1.x/ Typer: https://typer.tiangolo.com/
Currently, our drivers usually use argparse (or worse, nothing at all) to parse input arguments. This works ok, but requires explicit definitions for every configurable argument and can get quite messy. Even when set up well, using argparse to set up environments and configurations requires very long commands.
My suggestion is we build CLI tools using something like Click or Typer (python modules for CLI) that allow for loading an "environment" of sorts that can be modified with further commands. I think it would work well with the DAG runtime that I built a while ago.
An example I could see would be something like:
Of course, you could run multiple configuration set up commands, the from to . Do you want to continue?") which could be avoided with a
set
orupdate
commands could ask for confirmation of a change (e.g. "You are changing-y
command (like pip installs). Therun
command could include extra add-ons likeifarm
orpolaris
that would call special scripts to properly allocate GPUs/Nodes/etc on the respective machines.These behaviors are somewhat inspired from using things like the git and docker CLI systems which use sub-commands to access just a portion of the system.
Unfortunately, I don't have any projects that I can do this work for at the moment, so it may end up languishing here for a while. Just wanted to make sure the idea was captured in writing somewhere.