Module file with enviroment variables resolved at module load

[mvsurf]

It would be useful to have a way to specify env variable in modextravars which will be resolved when the module is loaded.

[boegel]

@mvsurfsara Can you give a specific example of a use case for this?

Please also mention how you worked this missing feature for now (i.e. via modluafooter)

[mvsurf]

@boegel Some applications require to set location of scratch or temporary directories to store temporary files (i.e. ADF SCM_TMPDIR in my specific case). This feature would make possible to set automatically these variable to user specific paths which can be resolved when the module is loaded (for example using $USER or $TMPDIR).

At the moment this can be achieved using modluafooter or modtclfooter.

In my case (we use Tcl module) I've used:

modtclfooter = "setenv SCM_TMPDIR $env(TMPDIR)"

This command injects the string between quotes into the module file as it is allowing to use module specific syntax.

[nordmoen]

It would be nice to have this feature when one package depends on another package, but wants to export its own variables.

Having modextravars = {'DEVICE_LIB_PATH': '$ENV{EBROOTCLANGMINAOMP}/amdgcn/bitcode'} export the correct Lua or TCL footer would be nice.

[ocaisa]

After playing around with it a bit, I think the regex to extract the environment variables from a string would be

[$]({?)[_a-zA-Z][a-zA-Z0-9_]*(}?)

which would match $HOME or ${HOME}. This would only work for simple replacements (i.e, not ${parameter:-defaultValue} for example), you'd need a check to ensure that if it starts with ${ it should end in }. You could then leverage this to replace the envvars with $::env{%(var)} or ' .. os.getenv(%(var)) .. ' as appropriate in the string.

If I was implementing this I would look at set_environment() and update_paths() for the two naming schemes. I'd extract environment variables in the values passed into a list, and replacing them with a magic string (XXXENVPLACEHOLDER) before passing the string to quote_str(). When it comes back, check the quoting character used (' or ") and then replace the placeholders in order with $ENV(%(var)) or %(quote) .. os.getenv(%(var)) .. %(quote)

[ocaisa]

Proof of concept:

import re
import copy

ENVVAR_REGEX = r'[$]({?)[_a-zA-Z][a-zA-Z0-9_]*(}?)'
ENVVAR_PLACEHOLDER ="XXXENVVARPLACEHOLDERXXX"

def find_envvars(txt):
    envvars = []
    matches = [x.group() for x in re.finditer(ENVVAR_REGEX, txt)]
    for envvar in matches:
        # Chomp the $
        envvar = envvar[1:]
        # Check for {
        if envvar.startswith('{'):
            # Make sure it ends with '}'
            if envvar.endswith('}'):
                envvar = envvar[1:-1]
            else:
                raise ValueError("Illegal environment variable %s in %s, only simple variables accepted (like $HOME or ${HOME})" %(envvar, txt))
        envvars.append(envvar)
    return envvars

def replace_envvars(txt, placeholder=ENVVAR_PLACEHOLDER):
    return re.sub(ENVVAR_REGEX, placeholder, txt)

def substitute_envvars(txt, envvars, placeholder=ENVVAR_PLACEHOLDER, lang='lua', quote='"'):
    txt_copy = copy.copy(txt)
    for envvar in envvars:
        if lang.lower() == 'lua':
            lang_envvar = quote + " .. os.getenv(%s) .. " % envvar + quote
        elif lang.lower() == 'tcl':
            lang_envvar = "$::env(%s)" % envvar
        else:
            raise ValueError("Language %s not supported, only Tcl or Lua!" % lang)
        txt_copy = txt_copy.replace(placeholder, lang_envvar, 1)

    return txt_copy

txt = "/path/to/my/$HOME ${PATH} $CHECK mymy"

envvars = find_envvars(txt)
substituted_txt = replace_envvars(txt)

#Stick it in quotes (to simulate our quoting)
substituted_txt = '"' + substituted_txt + '"'

print("Text is:\t", txt)
print("Envvars are:\t", envvars)
print("Lua syntax would be:\t", substitute_envvars(substituted_txt, envvars, lang='Lua', quote=substituted_txt[0]))
print("Tcl syntax would be:\t", substitute_envvars(substituted_txt, envvars, lang='Tcl'))

which outputs

Text is:     /path/to/my/$HOME ${PATH} $CHECK mymy
Envvars are:     ['HOME', 'PATH', 'CHECK']
Lua syntax would be:     "/path/to/my/" .. os.getenv(HOME) .. " " .. os.getenv(PATH) .. " " .. os.getenv(CHECK) .. " mymy"
Tcl syntax would be:     "/path/to/my/$::env(HOME) $::env(PATH) $::env(CHECK) mymy"