search

TACC / Lmod

Lmod: An Environment Module System based on Lua, Reads TCL Modules, Supports a Software Hierarchy
http://lmod.readthedocs.org
Other
489 stars 126 forks source link

Separator can cause cache syntax error #701

Closed vanderwb closed 3 months ago

vanderwb commented 6 months ago

Describe the bug If a module prepends (or appends, presumably) a path that ends with the separator character, it seems to cause a syntax error in the spider cache, which then causes a (non-fatal) error when running commands like module avail. Here is an example module command:

prepend_path("LD_LIBRARY_PATH", "/var/tmp/lib:")

Note the colon separator at the end of the path. The cache ends up with this entry for the module:

          lpathA = {
             = 1,
            ["/var/tmp/lib"] = 1,
          },

Since this isn't valid syntax, running commands which interrogate the cache produce this error:

bin/luac: .cache/lmod/spiderT.bug_x86_64_Linux.lua:24: unexpected symbol near '='

The prepend_path statement obviously isn't very well constructed, but it can happen, so if Lmod could handle this scenario better it would be great. I ran into this because the Spack package manager was auto-generating modules with a statement like the above. I will also submit a PR to resolve that particular problem with Spack.

To Reproduce I've attached a simple reproducer, to be executed with:

 $ env -i LMOD_ROOT=$LMOD_ROOT USER=$USER ./bug_report_template.sh

Desktop (please complete the following information):

$ module --miniConfig
                                                                                                                                                                                                                                                                                                                                                    Modules based on Lua: Version 8.7.37   2024-03-25 10:40 -06:00
    by Robert McLay mclay@tacc.utexas.edu

Changes from Default Configuration
----------------------------------

Name                         Where Set  Default                                                                                                  Value
----                         ---------  -------                                                                                                  -----
LFS_VERSION                  D          1.6.3                                                                                                    1.8.0
LMOD_AVAIL_STYLE             E          <system>                                                                                                 grouped:system
LMOD_CONFIG_DIR              E          /etc/lmod                                                                                                /glade/u/apps/spack-deployments/gust/23.09/envs/public/util
LMOD_MODULERC                E          /glade/u/apps/gust/23.09/spack/opt/spack/lmod/8.7.37/gcc/7.5.0/rohu/lmod/etc/rc                          /glade/u/apps/spack-deployments/gust/23.09/envs/public/util/hidden-modules
LMOD_PACKAGE_PATH            D          nil                                                                                                      /glade/u/apps/spack-deployments/gust/23.09/envs/public/util
LMOD_PAGER                   C          less                                                                                                     /usr/bin/less
LMOD_SITEPACKAGE_LOCATION    Other      /glade/u/apps/gust/23.09/spack/opt/spack/lmod/8.7.37/gcc/7.5.0/rohu/lmod/8.7.37/libexec/SitePackage.lua  /glade/u/apps/spack-deployments/gust/23.09/envs/public/util/SitePackage.lua
LMOD_SYSTEM_DEFAULT_MODULES  E          __unknown__                                                                                              ncarenv/23.09:craype/2.7.23:intel/2024.0.2:ncarcompilers/1.0.0:cray-mpich/8.1.27:netcdf/4.9.2
LMOD_SYSTEM_NAME             E          false                                                                                                    gust
LMOD_TCLSH                   C          tclsh                                                                                                    /glade/u/apps/gust/23.09/spack/opt/spack/tcl/8.6.12/gcc/7.5.0/katj/bin/tclsh
MODULEPATH_ROOT              E                                                                                                                   /glade/u/apps/gust/modules
PATH_TO_LUA                  C          lua                                                                                                      /glade/u/apps/gust/23.09/spack/opt/spack/lua/5.3.6/gcc/7.5.0/kflf/bin/lua

Where Set -> D: default, E: environment, C: configuration
             lmod_cfg: lmod_config.lua SitePkg: SitePackage StdPkg: StandardPackage
             Other: Set somewhere outside of normal locations
rtmclay commented 6 months ago

Thanks for the bug report with example test. This helps a great deal. Some packages do something special with an trailing delimiter. LUA env. vars do. So does tex/latex TEXINPUTS. So this cleaning up of trailing delimiters can not be done for all variables. So Lmod now removes extra delimiters for just PATH, LD_LIBRARY_PATH and MODULEPATH.

Please try the testing branch to see if it works for you. Thanks!

vanderwb commented 4 months ago

Apologies @rtmclay... I just realized I never got back to you after testing this! The change works great. Can LIBRARY_PATH be added to the list of vars to clean up? Other than that, it'd be great to get this into the next Lmod release.

rtmclay commented 4 months ago

Sure. cleaning up LIBRARY_PATH is now added and will be part of the next release.

rtmclay commented 3 months ago

Test out the release of Lmod 8.7.39 to see if this solves your issue.

rtmclay commented 3 months ago

Closing this issue. You can re-open this issue if necessary