Option to force non-cached load of specific modules when running spider

[mboisson]

Reported on the Lmod mailing list, but posting here for tracking.

The issue is that modules that modify MODULEPATH should not be loaded from the spider cache when module spider is run, since they can modify MODULEPATH dynamically. Such an example is to have them add a path in the user's home directory.

For example, our gcc/9.3.0 module has

prepend_path("MODULEPATH", pathJoin("/cvmfs/soft.computecanada.ca/easybuild/modules/2020", os.getenv("RSNT_ARCH"), "Compiler/gcc9"))
if isDir(pathJoin(os.getenv("HOME"), ".local/easybuild/modules/2020", os.getenv("RSNT_ARCH"), "Compiler/gcc9")) then
    prepend_path("MODULEPATH", pathJoin(os.getenv("HOME"), ".local/easybuild/modules/2020", os.getenv("RSNT_ARCH"), "Compiler/gcc9"))
end

module spider is fully aware that it does not know about pathJoin(os.getenv("HOME"), ".local/easybuild/modules/2020", os.getenv("RSNT_ARCH"), "Compiler/gcc9"), and if one sets MODULEPATH manually outside of this module (for example, with module use), it properly ignores the cache. However, since module spider loads the gcc/9.3.0 from the spider cache, it does not even "see" that this path, which did not exist at build time, now exists at run time.

A possible implementation is to let sysadmins set a module property or function which, if found, tells spider to not load from cache, but rather to execute that module:

add_property("lmod", "no_cache_spider_load")

Another implementation would be to systematically load any module which modifies MODULEPATH.

This is related to: https://github.com/ComputeCanada/software-stack/issues/110

[rtmclay]

I have updated the testing branch to support what I am calling "Dynamic MODULEPATH". The way it works is that Lmod now marks any modulefile that changes $MODULEPATH as special. This means that whenever a user runs module spider those special modules will re-evaluated even though they have been cached. This will typically be compiler and mpi modules in a software hierarchy.

To test this please do:

1. Install this version of Lmod from the testing branch
2. Rebuild the cache with this new version.
3. Use module spider to see if it finds the user's modulefiles in their local directory.

Please time this version and compare it to the results with the old version (With a user who doesn't have personal modulefiles) You can use the following to time both versions.

$ module --timer --no_redirect  spider 2> spider.log
$ tail spider.log

You will the times at the end of the file.

Site might have modulefiles that DO NOT change $MODULEPATH to a system path but might add a user directory in that module. So you will have to add haveDynamicMPATH() to your system modulefile:

local pkgVersion = "9"
local pkgName    = myModuleName()
local cmplr      = pathJoin("Compiler",pkgName,pkgVersion)
local hroot      = pathJoin(os.getenv("HOME") or "","myModules")
local userDir    = pathJoin(hroot,cmplr)

haveDynamicMPATH()

if (isDir(userDir)) then
   prepend_path("MODULEPATH",userDir)
end

Since the userDir will be a directory that root won't find, Lmod won't know that this modulefile has a dynamic $MODULEPATH. In this case, your modulefile will require haveDynamicMPATH() in order for Lmod to re-evaluate this module when producing the spider output.

To be clear I am looking for two things:

• Does this version work for you?
• Does this new version re-evaluating all modules that change $MODULEPATH when spidering expensive compared with the old way?

[mboisson]

Thanks Robert. I'll test this in the coming days.

[mboisson]

I still need to run some more tests, but the first one (modulepath in home, added by the module) seems to work.

We also add to MODULEPATH through the hooks, which I still have to test.

As for timing, here is what it looks like:

With our current version:

[mboisson@build-node ~]$ module --version; module --timer --no_redirect  spider 2> spider.log; tail -n 4 spider.log

Modules based on Lua: Version 8.6.16  2022-03-22 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Name         Count  T Time            Ave Time
----         -----  ------            --------
main         1      0.35187983512878  0.35187983512878
Cache:build  1      0.13766193389893  0.13766193389893

With the testing version:

[mboisson@build-node ~]$ module --version; module --timer --no_redirect  spider 2> spider.log; tail -n 4 spider.log

Modules based on Lua: Version 8.7.3 (8.7.3-12-gfec8199a) 2022-05-23 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Name         Count  T Time            Ave Time
----         -----  ------            --------
main         1      0.45551609992981  0.45551609992981
Cache:build  1      0.26619410514832  0.26619410514832

So, it is a tiny bit slower, but still significantly faster than without cache (even on a good filesystem):

[mboisson@build-node ~]$ export LMOD_RC=/home/mboisson/git/software-stack-config/lmod/lmodrc.lua
[mboisson@build-node ~]$ module --version; module --timer --no_redirect  spider 2> spider.log; tail -n 4 spider.log

Modules based on Lua: Version 8.6.16  2022-03-22 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Name         Count  T Time           Ave Time
----         -----  ------           --------
main         1      3.467894077301   3.467894077301
Cache:build  1      3.2546489238739  3.2546489238739

And this is with our very complex module structure, we have 188 modules which change MODULEPATH in our full hierarchy (just for one CPU architecture):

[mboisson@build-node ~]$ grep 'changeMPATH' /home/mboisson/.lmod/cache/avx2_intel/spiderT.lua  | wc
    188     564    5640

[mboisson]

Hi again, Ok, so it looks like it works with hooks too (this is what we use, by the way: https://github.com/ComputeCanada/software-stack-config/blob/main/lmod/SitePackage_localpaths.lua https://github.com/ComputeCanada/software-stack-config/blob/main/lmod/SitePackage.lua#L311

However, there is something very weird happening when I run module spider gsl/2.7 (that is the one module which I installed in my home, with gcc/9.3) with one version of gcc loaded....

module load intel/2020
module spider gsl/2.7   ==> ok, finds the version available with gcc/9.3, installed in my home directory
module load intel/2021
module spider gsl/2.7   ==> ok, finds the version available with gcc/9.3, installed in my home directory
module load gcc/9.3
module spider gsl/2.7   ==> ok, finds the version available with gcc/9.3, installed in my home directory
module load gcc/8
module spider gsl/2.7   ==> ok, finds the version available with gcc/9.3, installed in my home directory
module load gcc/10
module spider gsl/2.7   ==> Lmod has detected the following error:  Unable to find: "gsl/2.7".
module load gcc/11
module spider gsl/2.7   ==> ok, finds the version available with gcc/9.3, installed in my home directory

Our gcc modules are virtually identical (other than the version number)... it somehow fails only with gcc/10.... that's a bug that doesn't make sense to me...

[mboisson]

I figured out some more clues. The particularity of gcc/10 is that I have other things installed with gcc/10 in my home directory. To test this hypothesis, I copied

/home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/gcc9/gsl/2.7.lua

to

/home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/gcc10/gsl/2.8.lua

and

/home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/intel2020/gsl/2.10.lua

Now, I get:

module load gcc/8.3
module --terse spider gsl
...
gsl/2.7
gsl/2.8
gsl/2.10

module load gcc/9
module --terse spider gsl
...
gsl/2.7

module load gcc/10
module --terse spider gsl
...
gsl/2.8

module load intel/2020
module --terse spider gsl
...
gsl/2.10

It is as if if one of the modules with dynamic path is loaded (gcc/10 or gcc/9 or intel/2020) and some modules are found in there, it does not consider the other modules as having dynamic paths ?

[rtmclay]

You were correct. The version you tested did not know about user modules that were dynamic themselves. I was able to reproduce the problem and fix it for me. Please update your repo on the testing branch and test again.

[mboisson]

Mmm, no, I don't think that's it:

[mboisson@build-node ~]$ module --version

Modules based on Lua: Version 8.7.3 (8.7.3-24-gc431c184) 2022-05-23 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

[mboisson@build-node ~]$ module --quiet load gcc/8
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.7
gsl/2.8
gsl/2.10
[mboisson@build-node ~]$ module --quiet load gcc/9
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.7
[mboisson@build-node ~]$ module --quiet load gcc/10
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.8
[mboisson@build-node ~]$ module --quiet load intel/2020
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.10
[mboisson@build-node ~]$ module --quiet load intel/2021
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.7
gsl/2.8
gsl/2.10

Note that the gcc/8, gcc/9, gcc/10, intel/2020, intel/2021 modules are all modules that are installed centrally that have dynamic paths. They are not user-modules that have dynamic module paths.

However, within the dynamic module path that is added by those, no gsl exists for gcc/8 and intel/2021, but gsl/2.7 exists for the dynamic module path added by gcc/9, gsl/2.8 exists for the dynamic module path added by gcc/10, and gsl/2.10 exists for the dynamic module path added by intel/2020.

[mboisson]

Maybe this is more instructive:

[mboisson@build-node ~]$ module --quiet load gcc/8
[mboisson@build-node ~]$ echo $MODULEPATH | tr ':' '\n' | grep home
/home/mboisson/.local/easybuild/modules/2020/avx2/Core
/home/mboisson/.local/easybuild/modules/2020/Core
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.7
gsl/2.8
gsl/2.10
[mboisson@build-node ~]$ module --quiet load gcc/9
[mboisson@build-node ~]$ echo $MODULEPATH | tr ':' '\n' | grep home
/home/mboisson/.local/easybuild/modules/2020/avx2/MPI/gcc9/openmpi4
/home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/gcc9
/home/mboisson/.local/easybuild/modules/2020/avx2/Core
/home/mboisson/.local/easybuild/modules/2020/Core
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.7
[mboisson@build-node ~]$ module --quiet load gcc/10
[mboisson@build-node ~]$ echo $MODULEPATH | tr ':' '\n' | grep home
/home/mboisson/.local/easybuild/modules/2020/avx2/MPI/gcc10/openmpi4
/home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/gcc10
/home/mboisson/.local/easybuild/modules/2020/avx2/Core
/home/mboisson/.local/easybuild/modules/2020/Core
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.8
[mboisson@build-node ~]$ module --quiet load intel/2020
[mboisson@build-node ~]$ echo $MODULEPATH | tr ':' '\n' | grep home
/home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/intel2020
/home/mboisson/.local/easybuild/modules/2020/avx2/Core
/home/mboisson/.local/easybuild/modules/2020/Core
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.10
[mboisson@build-node ~]$ module --quiet load intel/2021
[mboisson@build-node ~]$ echo $MODULEPATH | tr ':' '\n' | grep home
/home/mboisson/.local/easybuild/modules/2020/avx2/Core
/home/mboisson/.local/easybuild/modules/2020/Core
[mboisson@build-node ~]$ module --terse spider gsl | grep "gsl/2.[7891]"
gsl/2.7
gsl/2.8
gsl/2.10
[mboisson@build-node ~]$ ls /home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/*/gsl
/home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/gcc10/gsl:
2.8.lua

/home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/gcc9/gsl:
2.7.lua

/home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/intel2020/gsl:
2.10.lua

[rtmclay]

Please provide a module tree that shows the issue and the output you expect.

[mboisson]

lmod_tree.tar.gz

Ok, attached is a minimalistic lmod tree, including:

• a "system" module tree
• a "in-user-home" module tree
• a lmodrc.lua which configures a path for a cache,
• said cache as generated with only the system modules in the MODULEPATH, i.e. with:

  [mboisson@build-node lmod_tree]$ echo $PWD
  /home/mboisson/lmod_tree
  [mboisson@build-node lmod_tree]$ echo $MODULEPATH_ROOT
  /home/mboisson/lmod_tree/system
  LMOD_RC=$PWD/lmodrc.lua $LMOD_DIR/update_lmod_system_cache_files $MODULEPATH_ROOT/Core

after generating the cache (to mimic a module folder which would be in a user's directory and which does not exists when the cache is created), I create a symlink in my home:

 ln -s lmod_tree/home/mboisson/ $HOME/modules

With the above, I get, without cache (i.e. note the LMOD_RC= before the module commands):

[mboisson@build-node lmod_tree]$ module list
No modules loaded
[mboisson@build-node lmod_tree]$ LMOD_RC= module --terse spider gsl
gsl/1.16
gsl/2.6
gsl/2.7
gsl/2.8
gsl/2.10

[mboisson@build-node lmod_tree]$ module load gcc/9
[mboisson@build-node lmod_tree]$ LMOD_RC= module --terse spider gsl
gsl/1.16
gsl/2.6
gsl/2.7
gsl/2.8
gsl/2.10

[mboisson@build-node lmod_tree]$ module load gcc/8

The following have been reloaded with a version change:
  1) gcc/9.3.0 => gcc/8.4.0     2) gcccore/.9.3.0 => gcccore/.8.4.0

[mboisson@build-node lmod_tree]$ LMOD_RC= module --terse spider gsl
gsl/1.16
gsl/2.6
gsl/2.7
gsl/2.8
gsl/2.10

while with cache, I get:

[mboisson@build-node lmod_tree]$ module list
No modules loaded
[mboisson@build-node lmod_tree]$ echo $LMOD_RC
/home/mboisson/lmod_tree/lmodrc.lua
[mboisson@build-node lmod_tree]$ module --terse spider gsl
gsl/1.16
gsl/2.6
gsl/2.7
gsl/2.8
gsl/2.10

[mboisson@build-node lmod_tree]$ module load gcc/9
[mboisson@build-node lmod_tree]$ module --terse spider gsl
gsl/1.16
gsl/2.6
gsl/2.7

[mboisson@build-node lmod_tree]$ module load gcc/10

The following have been reloaded with a version change:
  1) gcc/9.3.0 => gcc/10.3.0     2) gcccore/.9.3.0 => gcccore/.10.3.0

[mboisson@build-node lmod_tree]$ module --terse spider gsl
gsl/1.16
gsl/2.6
gsl/2.8

In other words, once I load a gcc module which includes a in-user-home MODULEPATH, it no longer finds modules in other in-user-home MODULEPATH.

[rtmclay]

Thanks very much for your example module tree! I am able to reproduce your issue. I do not have a solution yet. I'll let you know when I do.

[rtmclay]

Thanks again for your example module tree. It was a great help. The issue here was that an old optimization to walk or not walk was triggered when you loaded gcc/9.3.0 which prevented the complete walking of the module tree. This has been fixed.

This version now passes all my tests. Please test this version on the testing branch to see if it works for you

[mboisson]

Yes, that version fixes this. Unfortunately, it also seems to completely negate the performance improvement from the cache, regardless of whether there are actual user module files :(

Performance with the cache generated without dynamic module path info

$ module --version; module --timer --no_redirect  spider 2> spider.log; tail -n 4 spider.log

Modules based on Lua: Version 8.7.3 (8.7.3-33-gf14a7447) 2022-05-23 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Name         Count  T Time            Ave Time
----         -----  ------            --------
main         1      0.30467295646667  0.30467295646667
Cache:build  1      0.1292519569397   0.1292519569397

Performance with the dynamic module path info and the latest version:

$ module --version; module --timer --no_redirect  spider 2> spider.log; tail -n 4 spider.log

Modules based on Lua: Version 8.7.3 (8.7.3-33-gf14a7447) 2022-05-23 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Name         Count  T Time           Ave Time
----         -----  ------           --------
main         1      3.4761710166931  3.4761710166931
Cache:build  1      3.2802419662476  3.2802419662476

Performance without the cache:

$ module --version; module --timer --no_redirect  spider 2> spider.log; tail -n 4 spider.log

Modules based on Lua: Version 8.7.3 (8.7.3-33-gf14a7447) 2022-05-23 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Name         Count  T Time           Ave Time
----         -----  ------           --------
main         1      3.4012248516083  3.4012248516083
Cache:build  1      3.2089719772339  3.2089719772339

It seems that this version is loading every single module (not just those that have dynamic paths). With the cache generated with dynamic mpath info:

module --timer --no_redirect --trace  spider  |& grep "Spider Loading" | wc
   5079   25395  655976

With the cache generated without dynamic mpathinfo:

 module --timer --no_redirect --trace  spider  |& grep "Spider Loading" | wc
      0       0       0

[rtmclay]

I don't see this with my test cases. It is not "Spider Loading" any modules that I don't expect it to when I trace. Let's assume the case where there are no user modulefiles, If I have understood you, it is slower even if there are no user modules. Correct?

I have modified the tracing output to see which kind of Spider loading it is:

 running: module  -T --terse spider
  Building Spider cache for the following dir(s): OutputDIR/myModules4/Compiler/gcc9
  Spider Loading again: gcc/9.3.0 (fn: ProjectDIR/rt/dynamic_spider/mf4/Core/gcc/9.3.0.lua)
  Spider Loading again: gcc/10.3.0 (fn: ProjectDIR/rt/dynamic_spider/mf4/Core/gcc/10.3.0.lua)
  Spider Loading:       gsl/2.8 (fn: OutputDIR/myModules4/Compiler/gcc10/gsl/2.8.lua)
  Spider Loading:       gsl/2.7 (fn: OutputDIR/myModules4/Compiler/gcc9/gsl/2.7.lua)
  completed building cache. Saving cache: false

As you can see it is reloading the compiler modules and just loading the gsl modulefiles and only the ones in the user space.

1. So is it true that in the generated spider cache file there are module files that are marked with "changeMPATH=true" that shouldn't be?
2. Or are modulefiles in the being "Spider loading" (not again) that shouldn't be.

You will have to look at the output from module --trace spider to see which it is. And I'll need a module tree that shows it.

It seems to me that you should be able to use the different messages and the directories to know if some modulefile is being loaded that shouldn't be.

[mboisson]

I don't see this with my test cases. It is not "Spider Loading" any modules that I don't expect it to when I trace. Let's assume the case where there are no user modulefiles, If I have understood you, it is slower even if there are no user modules. Correct?

Correct, it is slower even if I remove the user modules.

I have modified the tracing output to see which kind of Spider loading it is:

 running: module  -T --terse spider
  Building Spider cache for the following dir(s): OutputDIR/myModules4/Compiler/gcc9
  Spider Loading again: gcc/9.3.0 (fn: ProjectDIR/rt/dynamic_spider/mf4/Core/gcc/9.3.0.lua)
  Spider Loading again: gcc/10.3.0 (fn: ProjectDIR/rt/dynamic_spider/mf4/Core/gcc/10.3.0.lua)
  Spider Loading:       gsl/2.8 (fn: OutputDIR/myModules4/Compiler/gcc10/gsl/2.8.lua)
  Spider Loading:       gsl/2.7 (fn: OutputDIR/myModules4/Compiler/gcc9/gsl/2.7.lua)
  completed building cache. Saving cache: false

As you can see it is reloading the compiler modules and just loading the gsl modulefiles and only the ones in the user space.

1. So is it true that in the generated spider cache file there are module files that are marked with "changeMPATH=true" that shouldn't be?
2. Or are modulefiles in the being "Spider loading" (not again) that shouldn't be.

You will have to look at the output from module --trace spider to see which it is. And I'll need a module tree that shows it.

It seems to me that you should be able to use the different messages and the directories to know if some modulefile is being loaded that shouldn't be.

Ok, I will test further with the new tracing output (but likely not before next week).

[mboisson]

Hi Robert, First, sorry if you received a notification from Github for a comment I just posted then deleted after I realized my mistake. Ignore that notification.

I believe the long delay I was seeing with the cache was caused by an incomplete cache. I regenerated the cache, and everything seems to work well and is quite quick.

With the old cache (without dynamic mpath info, quick but incorrect):

$ export LMOD_RC=/cvmfs/soft.computecanada.ca/config/lmod/lmodrc.lua:/cvmfs/soft.computecanada.ca/config/lmod/lmodrc/cache_avx2_intel.lua
$ module --version; module --timer --no_redirect  spider 2> spider.log; tail -n 4 spider.log

Modules based on Lua: Version 8.7.3 (8.7.3-40-gb465da7f) 2022-05-23 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Name         Count  T Time            Ave Time
----         -----  ------            --------
main         1      0.30733609199524  0.30733609199524
Cache:build  1      0.13697910308838  0.13697910308838

$ module --terse spider gsl
gsl/1.16
gsl/2.2.1
gsl/2.3
gsl/2.4
gsl/2.5
gsl/2.6

Without cache (slow and correct):

$ export LMOD_RC=/cvmfs/soft.computecanada.ca/config/lmod/lmodrc.lua
$ module --version; module --timer --no_redirect  spider 2> spider.log; tail -n 4 spider.log

Modules based on Lua: Version 8.7.3 (8.7.3-40-gb465da7f) 2022-05-23 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Name         Count  T Time           Ave Time
----         -----  ------           --------
main         1      3.3366219997406  3.3366219997406
Cache:build  1      3.1304070949554  3.1304070949554

$ module --terse spider gsl
gsl/1.16
gsl/2.2.1
gsl/2.3
gsl/2.4
gsl/2.5
gsl/2.6
gsl/2.7
gsl/2.8

and finally, with the updated cache with dynamic mpath info (quick and correct):

$ export LMOD_RC=/cvmfs/soft.computecanada.ca/config/lmod/lmodrc.lua:$HOME/.lmod/lmodrc_cache_avx2_intel.lua
$ module --version; module --timer --no_redirect  spider 2> spider.log; tail -n 4 spider.log

Modules based on Lua: Version 8.7.3 (8.7.3-40-gb465da7f) 2022-05-23 14:19 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Name         Count  T Time            Ave Time
----         -----  ------            --------
main         1      0.44126987457275  0.44126987457275
Cache:build  1      0.2593309879303   0.2593309879303

$ module --terse spider gsl
gsl/1.16
gsl/2.2.1
gsl/2.3
gsl/2.4
gsl/2.5
gsl/2.6
gsl/2.7
gsl/2.8

and here is the output with the new cache and trace on:

$ module --terse --trace spider gsl/2.7 |& grep -A1 -B1 home
running: module --terse --trace spider gsl/2.7
  Using Cache file: /home/mboisson/.lmod/cache/avx2_intel/spiderT.luac_5.1
  Building Spider cache for the following dir(s):
--
  dynamic spider search on /cvmfs/soft.computecanada.ca/easybuild/modules/2020/avx2/MPI/nvhpc20/cuda11.0/openmpi4
  dynamic spider search on /home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/gcc9
  dynamic spider search on /cvmfs/soft.computecanada.ca/easybuild/modules/2020/avx2/Compiler/gcc9
--
  dynamic spider search on /cvmfs/soft.computecanada.ca/easybuild/modules/2020/avx2/MPI/gcc11/openmpi4
  Full spider search on /home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/gcc10
  Spider Loading:       gsl/2.8 (fn: /home/mboisson/.local/easybuild/modules/2020/avx2/Compiler/gcc10/gsl/2.8.lua)
  dynamic spider search on /cvmfs/soft.computecanada.ca/easybuild/modules/2020/avx2/Compiler/gcc10

[rtmclay]

Great! I am glad that it is working for you. How do you feel about having automatic detection of modules that change $MODULEPATH versus requiring any dynamic modulefile to call haveDynamicMPATH()?

[mboisson]

Detecting it automatically is definitely a lot more convenient since we don't have to modify existing modules, but haveDynamicMPATH() could be handy in some corner cases. I don't think we have any such case at this time however.

[mboisson]

However, I guess it comes at a small performance cost, so sites which don't have such modules may want to turn this off. It could be detected automatically, but only acted upon if some configuration or environment variable is set ?

[rtmclay]

Closing this issue. This feature has been merged into Lmod 8.7.4