rcs-si / module_check

Check if environment variables within a module file are defined correctly
2 stars 0 forks source link

0.5 bugs #10

Closed bu-bgregor closed 8 months ago

bu-bgregor commented 9 months ago

This works (without loading it as a module):

/share/pkg.8/module_check/0.5/install/module_check/module_check -c python3/3.10.12

Now load it as a module:

module use /share/module.8/rcstools
module load module_check/0.5
module_check -c python3/3.10.12

..but...module_check throws errors based on its own module being loaded when testing :

Exception: *** Environment variable incorrect ***
SCC_MODULE_CHECK_DIR

So you'll need to filter out all the "SCC_MODULECHECK*" strings from the environment variables before comparing the file vs. env variables.

Also, we need to figure out how to support modules that require other modules. As an example, quantumespresso/7.2 requires the openmpi/4.1.5_gnu-12.2.0 module:

$ module load quantumespresso/7.2

-------------------------------------------------------------------------------
ERROR: quantumespresso/7.2 requires at least one additional module.
Run the following commands to load all of the dependencies:

module load openmpi/4.1.5_gnu-12.2.0
module load quantumespresso/7.2
-------------------------------------------------------------------------------

module_check correctly returns that error if the openmpi module is not loaded:

$ /share/pkg.8/module_check/0.5/install/module_check/module_check -c quantumespresso/7.2
Traceback (most recent call last):
  File "/share/pkg.8/module_check/0.5/install/module_check/module_check.py", line 71, in <module>
    main()
  File "/share/pkg.8/module_check/0.5/install/module_check/module_check.py", line 59, in main
    check_module_loadable(modname)
  File "/share/pkg.8/module_check/0.5/install/module_check/module_check.py", line 17, in check_module_loadable
    raise Exception(msg + stderr)
Exception: *** Module could not be loaded ***

-------------------------------------------------------------------------------
ERROR: quantumespresso/7.2 requires at least one additional module.
Run the following commands to load all of the dependencies:

module load openmpi/4.1.5_gnu-12.2.0
module load quantumespresso/7.2
-------------------------------------------------------------------------------

But...if I first load openmpi then run module_check it finds the SCC_OPENMPI variables in the environment but not in the quantumespresso modulefile so it throws an error:

$ module load openmpi/4.1.5_gnu-12.2.0
[bgregor@scc1 module_check]$ /share/pkg.8/module_check/0.5/install/module_check/module_check -c quantumespresso/7.2
Traceback (most recent call last):
  File "/share/pkg.8/module_check/0.5/install/module_check/module_check.py", line 71, in <module>
    main()
  File "/share/pkg.8/module_check/0.5/install/module_check/module_check.py", line 63, in main
    check_module_env(modname) 
  File "/share/pkg.8/module_check/0.5/install/module_check/module_check.py", line 34, in check_module_env
    raise Exception(msg + comparison_result)
Exception: *** Environment variable incorrect ***
SCC_OPENMPI_INCLUDE

Maybe when modulecheck.py first starts up it should check the environment for any SCC* variables, and then when you retrieve env vars after loading the module being tested filter out the ones found at startup.

ex.: with the module_check module loaded you'll find SCC_MODULE_CHECK_DIR, SCC_MODULE_CHECK_BIN. With the openmpi module you'll also find SCC_OPENMPI_INCLUDE and a bunch more. You can then pass a list of those variables into the get_module_env_vars() function and then remove them from the list of env vars loaded in the function call. Then for quantumespresso you'll just have the SCCQUANTUMESPRESSO* variables to compare against the output of module help I think this will fix the behavior when module_check is loaded as a module and when there are other modules that need to be loaded.

SDallmn commented 8 months ago

When parsing stderr to identify module help env variables, can accidently pick up incorrect variables if the text in help has similar formatting to SCC_* variables at bottom of help string.

bu-bgregor commented 8 months ago

Fixed by Seamus.