search

TACC / Lmod

Lmod: An Environment Module System based on Lua, Reads TCL Modules, Supports a Software Hierarchy
http://lmod.readthedocs.org
Other
471 stars 122 forks source link

Module switch while maintaing order #616

Closed servesh closed 1 year ago

servesh commented 1 year ago

When one swaps modules, it would be helpful if there was a way to replace the relevant paths of the particular module is replaced in place maintaining path order.

module load foo/0.1
LD_LIBRARY_PATH=/opt/bar/0.1/lib:/opt/foo/0.1/lib:/opt/baz/0.1/lib

module switch foo/0.1 foo/0.2 does this, LD_LIBRARY_PATH=:/opt/foo/0.2/lib:/opt/bar/0.1/lib:/opt/baz/0.1/lib

instead of LD_LIBRARY_PATH=/opt/bar/0.1/lib:/opt/foo/0.2/lib:/opt/baz/0.1/lib

Any thoughts on how this can be supported?

rtmclay commented 1 year ago

It would be extremely difficult to make this work. A module swap (module switch is just an alias for swap) unloads the first module and then the second is loaded. Lmod only knows that there are two modules to swap:

$ module swap A B

It doesn't know about the contents of each modules. The Lua interpreter is evaluating each modulefile separately. So by the time that "B" is being loaded A's prepend_path's entries are gone. There is no way for Lmod to know the previous order in LD_LIBRARY_PATH.

Also this violates a fundamental principal of Lmod. All commands are done as asked for. In module B, you asked for prepend_path(). That means that the entry is placed first.

Now if you know that you always want "/opt/bar/..." first then you can use the priority feature of prepend_path: See https://lmod.readthedocs.io/en/latest/077_ref_counting.html?highlight=priority#specifying-priorities-for-path-entries for details. However, this not a general solution.

Feel Free to submit a PR to support this as an optional way to handle swaps

servesh commented 1 year ago

I wasn't aware that swap or switch is nothing but a unload/load in one command. I was assuming, some amount of info is known when swap or switch is used unlike load/unload, i.e. when the prior modulefile's paths which are scanned and removed, the index at which these changes occur can be used as the location to update the new module's paths into.

rtmclay commented 1 year ago

The point I was making above was that. Lmod is not evaluating the modules Lua is. Lmod is there to handle the setenv(), prepend_path() etc functions. Lmod has no idea of what the contents of the modulefiles until Lmod functions are called.

I'm closing this issue.