Closed sajith closed 9 months ago
Also there are several manual (and therefore error-prone) steps involved in configuring FABlib for the first time. Perhaps we can have a small cli for these tasks, such as fablib configure
and maybe fablib update-tokens
and the such?
To give credit where it is due: this idea from Microsoft Planetary Computer's Python SDK, which offers a planetarycomputer configure
command.
Perhaps we could use python-dotenv instead of configparser.
Do we still want fabric_rc
to bash environment file?
I think it would be good to keep it as an option for backward compatibility. Hopefully, there is a bash/env reader library we can use. Then it could default to a newer config style (.yml ?) but fall back to fabric_rc
if that is all that is found.
Perhaps we can add a command like fablib-cli env
(or pull fabric-cli
into fablib repository and add an env
subcommand to it) that will emit exports the way ssh-agent
does:
$ ssh-agent -s # for bash
SSH_AUTH_SOCK=/tmp/ssh-XXXXXXGrSGCL/agent.1103238; export SSH_AUTH_SOCK;
SSH_AGENT_PID=1103239; export SSH_AGENT_PID;
echo Agent pid 1103239;
or, in the case of csh like shells:
$ ssh-agent -c
setenv SSH_AUTH_SOCK /tmp/ssh-XXXXXXfgyMqr/agent.1103652;
setenv SSH_AGENT_PID 1103653;
echo Agent pid 1103653;
That way eval $(fablib-cli env)
should add the environment variables to the current shell. We might be breaking only some compatibility if we remove fabric_rc
, but not all.
FABlib's configuration file is something of an odd duck for three reasons: (1) because of its location, and (2) because of its contents, and (3) because of order of precedence in which configuration file overrides environment variables (usually it is the other way around).
First, the location: the configuration file is expected to be found in
~/work/fabric_config/fabric_rc
:https://github.com/fabric-testbed/fabrictestbed-extensions/blob/3f8c55be24afb04adb07a54b8ee1db88aae9acd1/fabrictestbed_extensions/fablib/fablib.py#L532
This is different from (for example)
~/.config/fabric-testbed.net/
on Linux distros, and~/Library/Application Support/fabric-testbed.net
on macOS, and whatever similar thing they might have on Windows. The library platformdirs can help us with this, and platformdirs is already an indirect dependency for us because jupyter-core depends on it.Second, format of the configuration file:
Good thing about this format is that we can
source
it from bash, although maybe not from other shells. Bad thing is that it is a bit of an odd format for a configuration file, and we use hand-rolled code to parse it:https://github.com/fabric-testbed/fabrictestbed-extensions/blob/3f8c55be24afb04adb07a54b8ee1db88aae9acd1/fabrictestbed_extensions/fablib/fablib.py#L541-L552
Perhaps we can use Python's configparser to get it to a less bash-like shape, and avoid having a custom parser? And then the config file would look like:
The current configuration style works well enough, and it is simple and understandable, and we have an existing workflow when onboarding new FABRIC users. I am wondering if we should look at making it a little more "standard-ish", and if it is worth the trouble.
The third thing is that environment variables are usually used to quickly override settings from configuration files, and not the other way round.
Now, none of the above-stated three things are deal-breakers, and there's more urgent work to be done in fablib. There's also that changing the existing behavior has costs.
If we try to change the existing configuration method, I imagine that the fallout will involve: (1) updating the
configure.ipynb
notebook accordingly, (2) communicating the change to FABRIC users early, (3) having some way to migrate the existing configurations, both in FABRIC-hosted JupyterHub instance and elsewhere, and (4) we will also need to decide if we're going to break backward compatibility or not, and if we do want to break it, how to go about it.I just wanted to write down my thoughts (mostly because I noticed that we are doing things our special way, and I'm not sure if it is a good thing or a bad thing),
do some bikesheddingcollect some thoughts, and, in the end, if we do end up making any changes, ensure that we make them in a considered manner.