This pr adds a few flags for processing by the server creation engine before it executes

The server config can have the following additional args

env_vars: Optional environ variables that will be copied as is from user environment to execution environment
required_env_vars: Mandatory environ variables that will be checked for existence before being copied to execution environment
env_mounts: A user defined list mapping from <STR | ENV_VAR>:<STR | ENV_VAR>. The engine will resolve the ENV_VAR at runtime, and consider it a mandatory argument to be passed by the user.

This enables for dynamic mounting by the user without the hardcoding of local and slurm mount paths - all users can share the same config file, and simply populate the mount paths via personal environment variables provided in their private .bashrc or .profile (or even explicit export).

This will reduce the number of copy pasted user-defined versions of the example server configs and prevent users from having to manually maintain and update their configs as and when the core repo updates with newer containers.

Example

# cluster_configs/local.yaml
...

# optional env variables
env_vars: ['NEMO_SKILLS_CONFIG']
# required env variables
required_env_vars: ['HF_HOME', 'HF_TOKEN']

# mounting determined by environment variables
env_mounts:
  - "{HF_HOME}:/cache/huggingface"

Kipok / NeMo-Skills

Add support for env variable passthrough and env variable mounting #129

Example