search

Matgenix / jobflow-remote

jobflow-remote is a Python package to run jobflow workflows on remote resources.
https://matgenix.github.io/jobflow-remote/
Other
18 stars 11 forks source link

Set `projects_folder` via env var #188

Open janosh opened 18 hours ago

janosh commented 18 hours ago

Small QoL suggestion: would be nice if the projects_folder which currently can only be configured in ~/.jfremote.yaml could also be set via env var JF_REMOTE_PROJECTS_FOLDER. one motivation for this is that some HPC setups don't provide access to the home folder from compute nodes

https://github.com/Matgenix/jobflow-remote/blob/8df197923b77862035f893d58d1cf1e022869cf1/src/jobflow_remote/config/settings.py#L19-L21

gpetretto commented 17 hours ago

If I understand correctly your request, this should already be possible, since the settings are a pydantic BaseSettings: https://docs.pydantic.dev/latest/concepts/pydantic_settings/#environment-variable-names

For example I regularly set my default project with export jfremote_project=$PROJECT_NAME. In principle you could also change the path of the config file ~/.jfremote.yaml by setting the environment variable jfremote_config_file and have it in some other path than the home.

Does this solve your issue?

janosh commented 14 hours ago

thanks for the quick reply! this was what i was looking for

export JFREMOTE_PROJECTS_FOLDER=/dir/to/jf_remote/configs

would be great to document the env var prefix JFREMOTE_ for any JobflowRemoteSettings attribute

gpetretto commented 4 hours ago

Actually it is already documented: https://matgenix.github.io/jobflow-remote/user/projectconf.html#general-settings I admit it may not be that easy to spot. If you have a suggestion of where to move it to make it more reachable I can update it.

janosh commented 42 minutes ago

i remember now seeing the docs you linked before. just forgot about them. my bad!

i think that's already the best place to document them. if you search for "environment variable" on that page, it only comes up once at the top. the term isn't used again at the bottom where the variable names are actually used. i think adding a subsection titled "Environment Variables" listed under "On this page" would help discoverability. and CONSTANT_CASING the variable names could also help since that's what i'm looking for when skimming the page