Closed boegel closed 9 years ago
as regards this one, I can see 3 possible directions: 1) cultivate a symlink farm - yet, consider dependency resolution issues inside the modulefiles 2) use aliases - ie. dependent modules would come up with the easybuild namespace in their version strings 3) define a mapping function - this requires perfect unambiguous 1-1 mapping for proper functionality
I find that the 1st one is trivial to implement -without the deps- outside eb mechanisms; while with the deps, it will result in complex modulefiles. So, not too attractive.
The 2nd is kind of middleground to allow some pretty good compatibility with sites making the transition from legacy module namespace schemes to easybuild (which is far more rigorous)
The 3rd one is a more generic approach and should cover multiple scenarios, in effect putting the responsibility on the sysadmins to provide a consistent namespace.
The mapping function could also include retrieving the info from a "module mapping file", which allows infinite freedom and the possibility to open any can of worms you fancy...
Hi there,
after pondering a bit more on this one, here's some summary of what I think is the big picture:
category1/category2/.../categoryN/package/version/spec1/spec2/.../specN
Because we cannot have it all before v1.0 -due to time needed to implement the "best" solution-, one tractable approach to follow could perhaps be a "mapping" file (ie. a table of 2 columns) This becomes a python dictionary to assist with the production of the alternative namespace. (but not sure if this should rather be invoked as a separate secondary step)
Hopefully that covers the following scenarios relatively easily (read: before v1.0), and until we understand what the perfect solution is really about:
module load netcdf/intel/64/4.0
module load netcdf/4.1.1/pgi/10.3/64
module load netcdf/1.4.3-mpich2-intel-64bit # doable right now with suffix?
module load misc-libs/netcdf/3.6.3_intel # doable right now with suffix & moduleclass?
module load package/version/arch/fpaccuracy
In the ideal (long-term) situation, the full flexibility of modules should be available, with multiple categories being possible, for example:
bioinformatics/tools/MrBayes/3.2/intel/12.0/64
I have to admit I cannot easily grasp all the potential different specializations of environment modules configuration, therefor I suggest to go for now with the file-mapping approach described above, which should be the most future-proof and saves us from bothering with all the fancy (and often crippled) namespace schemes.
In essence, the proposed idea is option (3) of the previous email in this thread.
Fotis
Considering my remark on #346, I do not think that the splitting should be done in the easyconfigs. That would mean each site with a different naming scheme would have to change all the easyconfigs they want. I hardly see the problem with offering a class that splits the module name into some format easybuild can handle and allow sites to offer a subclass that changes this to their needs.
No, not in the easyconfigs, that would be madness.
In the EasyBuild configuration file, where you also specify things like log_format
and install_path
, see https://github.com/hpcugent/easybuild/wiki/Configuration.
My current plan is to support a function like construct_module_name
or something, that a user can define. That function would then take an EasyConfig
instance, which has all the possible info, and can spit out a string/list/dictionary (not sure yet), that defines the corresponding module name.
If such a function is defined, EasyBuild would use it when generating modules.
The current default would then be equivalent with something like (untested):
def construct_module_name(ec):
return [ec.name, ec.get_installversion()]
You'd probably also need a function that does the reverse, i.e. deconstruct a given module name in parts and make sense of them.
shouldn't we close this one now that v1.8.0 is out?
I'll consider this fully fixed one #687 is handled, so let's leave it open for now.
(old internal ticket 284, 324)
We should allow a way to specify a custom naming scheme for modules, instead of the default that EasyBuild uses now, i.e.
name-prefix-version-toolkit-suffix
.This is badly needed for most big HPC sites.
Notes, as discussed during the 1st EasyBuild hackathon:
The list below gives an overview of what this custom naming scheme should support (compiled by @fgeorgatos):