Open Micket opened 3 years ago
To clarify:
There are 2 dimensions on this (or 2 issues):
First is the treatment of the --installpath <prefix2>
when having multiple module trees:
MODULEPATH=<prefix1>/all/Core
<prefix2>/all
Core/GCC/10.2.0
which fails to find those modules from <prefix1>
--> For this options 1&2 are suitable: Either (also) check for GCC/10.2.0
or in step 2 above also add <prefix1>/all
to MODULEPATH since it ends with /Core
The second issue is a problem with HMNS in general when having multiple trees: In the example above installing a compiler module into <prefix1>
will only expand MODULEPATH in <prefix1>
, but not in <prefix2>
Although (assuming) MODULEPATH=<prefix1>/all/Core:<prefix2>/all/Core
is set.
For that Solution 3 would be great: Currently it is only expanding the single path or relying on EASYBUILD_ENVVARS_USER_MODULES
and such which requires users(!) to export env variables with module trees and admins to (re)build modules to support that. To avoid this the module file could extend the MODULEPATH by all existing folders with the new suffix. E.g. for each <path>
in $MODULEPATH that ends in /Core
check for <path>/../<suffix>
and add that to MODULEPATH
You can now extend MODULEPATH in any direction you want from the system installation via the use of environment variables. We are using that configuration at JSC for user installations, which is provided as a module. The settings we use at JSC are
--subdir-user-modules=easybuild/$SYSTEMNAME/modules --envvars-user-modules=PROJECT,HOME
where $SYSTEMNAME
is a default envvar set per system. The one thing that this will not give you is any builds the user makes using SYSTEM
toolchain since the userpath to all/Core
is not in MODULEPATH
(although you could do this in the user-facing module).
If you'd like you can take a look at our module at https://gist.github.com/ocaisa/b13f243936f6de0b434edd9208ca1c76 . The basic stuff is
-- Set up exactly where we put our installations
local subdir_user_modules = pathJoin("easybuild", systemname)
local home_eb = pathJoin(os.getenv("HOME"), subdir_user_modules)
local project_eb = nil
local install_eb = home_eb
local home_install_eb = true
if os.getenv("PROJECT") then
project_eb = pathJoin(os.getenv("PROJECT"), subdir_user_modules)
if not os.getenv("PREFER_USER") then
install_eb = project_eb
home_install_eb = false
if mode()=="load" then
LmodMessage(yellow.."\nFound $PROJECT in the environment, using that for installation. To override this and do \n"..
"a personal installation set the environment variable PREFER_USER=1 and reload this module.\n"..normal)
end
end
end
-- ensure this is a symlink since there is very little room in HOME directories (and give advice if it isn't)
if mode()=="load" and home_install_eb then
if lfs.symlinkattributes(pathJoin(os.getenv("HOME"),"easybuild"), "mode") ~= "link" then
LmodError(yellow.."\nPlease configure ~/easybuild to be a symlink to a location with significant data quotas,\n"..
"for example with: \n mkdir -p $PROJECT/$USER/easybuild && ln -s $PROJECT/$USER/easybuild $HOME/easybuild"..normal)
else
LmodMessage(yellow.."\nPerforming a personal installation. To do a project wide installation, \n"..
"set the $PROJECT environment variable (for example via jutil).\n"..normal)
end
end
To get around the issue of MODULEPATH
and Core
we use an alias
-- Configure an alias for the eb command so that modules in JSC tree are found by eb
-- (it searches for them based on the prefix)
local stage_path = pathJoin(stages, stage)
local stage_base_moduledir = pathJoin(stage_path, "modules/all")
local stage_base_moduledir_core = pathJoin(stage_base_moduledir, "Core")
set_alias("eb","MODULEPATH=" .. stage_base_moduledir .. ':' .. stage_base_moduledir_core .. ':$MODULEPATH STAGE=' .. stage .. " eb")
I should say, we use hooks to disallow that user builds things that extend the module path (compiler, mpi), instead only using the toolchains that we provide (though this can be overridden).
I should say, we use hooks to disallow that user builds things that extend the module path (compiler, mpi), instead only using the toolchains that we provide (though this can be overridden).
Another use case: I'm regularly testing ECs and EBs using the system installations as a base but also install compilers (or more often MPIs) into a "user" module-tree. So even for devs this whole thing is a hassle.
And yes something like your module works (thanks for that) but I think with the proposed changes we could provide first-class support for HMNS and make it "just work".
For sure option 1 would avoid the need for the alias, which I think would be a good move and should not be very difficult to support (I imagine).
Another use case: I'm regularly testing ECs and EBs using the system installations as a base but also install compilers (or more often MPIs) into a "user" module-tree. So even for devs this whole thing is a hassle.
This is only relevant where you are overriding a compiler/mpi available on the system (or you intend to in the future), otherwise the branch would be completely unpopulated. In that case you can still leverage the environment variable approach, but from the user installations module. I can imagine something like
--subdir-user-modules=easybuild/modules --envvars-user-modules=SYS_INSTALL
were
SYS_INSTALL
is evaluated and exported in the user module.
Yes, I don't say it is impossible, just that it currently is quite some trouble and things to understand until it works. And that we should improve upon that :)
The main problem with basing things on the subdir-user-modules is that it locks down the prefix to be $HOME.
With project storage, something we and other sites use, one would want to use some freely choosen prefix, like /proj/nobackup/some-project/easybuild for instance.
Doing that excludes the usage of subdir-user-modules.
But a combination of options 1 and 3 should be able to handle this.
The main problem with basing things on the subdir-user-modules is that it locks down the prefix to be $HOME.
--envvars-user-modules
can be set to a list of env vars containing the prefix, so $HOME is only the default, you can set it as you like hence your use case can already be realized.
The main problem with basing things on the subdir-user-modules is that it locks down the prefix to be $HOME.
--envvars-user-modules
can be set to a list of env vars containing the prefix, so $HOME is only the default, you can set it as you like hence your use case can already be realized.
@Flamefire is right, see https://github.com/easybuilders/easybuild-framework/issues/3703#issuecomment-849418800 above for the exact use case you (@akesandgren) describe
Ah ok, didn't remember how those changes affected the use of that...
Currently, EB won't find anything from your normal MODULEPATHs, only stuff that's in
--installpath-modules
. Not to bad for sysadmins to work around (just expand the MODULEPATH to also include the "/all" module directory directly, but it gets painful when lettings users build their stuff (especially if they use multiple paths as well). t would be nicer to just deduce these from the normal MODULEPATH's instead.From the zoom discussion, some options presented were
I think option 1 seems most promising, with a configuration option to disable it. Heck, it can even be off by default since it would be simple to enable for those who use this so it would be perfectly safe feature to add.
@Flamefire ping (since you are also interested in this).