Getting LUA_PATH and LUA_CPATH at configure time

[t-brown]

If a user sets their own LUA_PATH/LUA_CPATH and does not include a double semi-colon (;;) at the end of it lua will no longer look at it's default path. This means Lmod is unable to find dependent modules, such as posix.

For example with no LUA_PATH (everything works)

$ echo $LUA_PATH $LUA_CPATH

$
$ lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> print(package.path)
./?.lua;/usr/share/lua/5.1/?.lua;/usr/share/lua/5.1/?/init.lua;/usr/lib64/lua/5.1/?.lua;/usr/lib64/lua/5.1/?/init.lu
> print(package.cpath)
./?.so;/usr/lib64/lua/5.1/?.so;/usr/lib64/lua/5.1/loadall.so

If the user sets LUA_PATH to /tmp (for example).

$ export LUA_PATH='/tmp/?.lua'
$ export LUA_CPATH='/tmp/?.so'
$ ml av
/usr/bin/lua: error loading module 'posix' from file '/tmp/':
        /tmp/: cannot read file data: Is a directory
stack traceback:
        [C]: ?
        [C]: in function 'require'
        ...rh6/software/lmod/6.0.8/libexec/../tools/fileOps.lua:35: in main chunk
        [C]: in function 'require'
        ...x86_64/rh6/software/lmod/6.0.8/libexec/myGlobals.lua:40: in main chunk
        [C]: in function 'require'
        ...rc/tools/x86_64/rh6/software/lmod/6.0.8/libexec/lmod:76: in main chunk
        [C]: ?

If the user sets LUA_PATH to /tmp correctly.

$ export LUA_PATH='/tmp/?.lua;;'
$ export LUA_CPATH='/tmp/?.so;;'
$ ml av

------------------------------------------------ Compilers -------------------------------------------------
   gcc/5.1.0    intel/15.0.2 (m)    pgi/15.3

----------------------------------------- Independent Applications -----------------------------------------
   allinea/forge/5.0.1 (m)      idl/8.5                paraview/4.3.1          stdenv
   allinea/forge/5.1   (m)      itac/9.0.3.051  (m)    paraview/5.0.0 (D)      subversion/1.8.13
   allinea/forge/6.0   (m,D)    jdk/1.8.0              pdtoolkit/3.20          tcltk/8.5.18
   autotools/2.69               loadbalance/0.1        perl/5.22.0             tcltk/8.6.4       (D)
   cmake/3.2.2                  matlab/2015a           qt/5.5.0                totalview/8.15.4
   cuda/7.0.28         (g)      matlab/2015b    (D)    rocoto/1.2              udunits/2.2.19
   curc-bench/master            ncl/6.3.0              ruby/2.2.3              valgrind/3.10.1
   expat/2.1.0                  nodejs/4.2.4           slurm/blanca   (S)      visit/2.10.0
   git/2.4.2                    papi/5.4.1             slurm/slurm    (S,D)

------------------------------------------ Lmod Internal Modules -------------------------------------------
   lmod/6.0.8    settarg/6.0.8

  Where:
   S:  Module is Sticky, requires --force to unload or purge
   g:  built for GPU
   m:  built for host and native MIC
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

One idea would be to modify the luaModuleAvailable program in configure.ac to try and find the package and/or .so and then populate src/lmod.in.lua as is currently done to find the lmod packages.

[rtmclay]

I have created 6.2.2 on github. This version protects Lmod from user changes to LUA_PATH and LUA_CPATH. Or at least it does for me. When you get a change please test this out at your site.

This new version uses the technique that Tim suggested. It uses the values of LUA_PATH and LUA_CPATH at the time of configuration for the installation of Lmod. This means that sites will have to use LMOD_PACKAGE_PATH to let Lmod know where SitePackage.lua. is. They can't change the value of LUA_PATH after Lmod is installed to find it.

[wpoely86]

I think this broke the configure on my system. The script fails to find posix:

checking for lua modules: posix lfs
/usr/bin/lua: error loading module 'posix' from file '/usr/lib64/lua/5.3/posix.so':
    /usr/lib64/lua/5.3/posix.so: undefined symbol: luaopen_posix
stack traceback:
    [C]: in ?
    [C]: in function 'require'
    luaModuleAvailable:3: in function 'main'
    luaModuleAvailable:10: in main chunk
    [C]: in ?

Running /usr/bin/lua luaModuleAvailable posix myself works just fine. Neither LUA_PATH or LUA_CPATH is definied.

[wpoely86]

Confirmed: git bisect says 12cf882210acce8ba14ed8c07ea25ca4234ec007 is the faulty commit.

[wpoely86]

Works for me:

diff --git a/configure b/configure
index c842b11..1989e6a 100755
--- a/configure
+++ b/configure
@@ -3710,7 +3710,11 @@ echo "... $LuaV"

 PATH_TO_LUA=$(dirname $luaprog)

-export LUA_PATH=$ORIG_LUA_PATH
+if [ -z $ORIG_LUA_PATH ]; then
+    unset LUA_PATH
+else
+    export LUA_PATH=$ORIG_LUA_PATH
+fi

[rtmclay]

Thanks for the careful testing and the fix. I learned yet again the difference between having unset environment variable versus having LUA_PATH="". I have pushed a new version of Lmod (6.2.4).

Please test this version out for me when you get a chance.

Thanks, R.

[wpoely86]

@rtmclay Works again. Thanks!

[opoplawski]

New wrinkle (I can open a new issue if you want). For Fedora Lmod packages we can build on any arch - so LUA_PATH can reference either /usr/lib/lua/... or /usr/lib64/lua/... This then gets encoded into Lmod and so breaks it running on a different architecture. I can fix it by making Lmod an arch specific package, which may be the easiest thing to do.

[rtmclay]

I think that it will have to be an arch specific package. But I'm not completely sure that it is necessary. Why can't you set:

 export LUA_PATH="/usr/lib/lua/5.1/?.lua;/usr/lib64/lua/5.1/?.lua

As long as only one directory is filled, this will work.

R.

[opoplawski]

You can have both the 64-bit and 32-bit lua libraries installed at the same time. So yeah, going arch specific seems to be the way to go here.