easybuilders / easybuild-framework

EasyBuild is a software installation framework in Python that allows you to install software in a structured and robust way.
https://easybuild.io
GNU General Public License v2.0
148 stars 201 forks source link

Wrong module loaded in HMNS: Wrongly using NOT the SYSTEM module #4619

Open Flamefire opened 2 weeks ago

Flamefire commented 2 weeks ago

I noticed this when trying to reinstall flex-2.6.4-GCCcore-12.2.0.env:

The relevant part of the EC are this:

toolchain = {'name': 'GCCcore', 'version': '12.2.0'}
toolchainopts = {'pic': True}

builddependencies = [
    ('Bison', '3.8.2'),
    ('help2man', '1.49.2'),
    # use same binutils version that was used when building GCC toolchain
    ('binutils', '2.39', '', SYSTEM),
]

What now happens during the build is:

However for HMNS loading GCCcore brings the binutils-GCCcore into view such that the subsequent module load binutils/2.39 loads the wrong module!

ocaisa commented 2 weeks ago

This is because a hierarchical MNS is not guaranteed unique. Given that EasyBuild supports custom MNSes in general, and those may not be unique, I don't see an easy way out of this problem.

My only suggestion for framework would be that we build 2 MNS trees when using a non-default module tree. The main build is done with the default MNS, and then rebuild the module using the target MNS. That way we sidestep these issues in the build...but they are still present in the hierarchical module tree for end users.

akesandgren commented 2 weeks ago

We could workaround that by loading any SYSTEM module BEFORE the toolchain...

Flamefire commented 2 weeks ago

We could workaround that by loading any SYSTEM module BEFORE the toolchain...

I like that solution. Might need some work in framework but I can't see this causing issues, at least with any more or less standard toolchain.
So it is logically sound although not fully trivial to implement

branfosj commented 2 weeks ago

Would the Lmod module swap functionality attempt to swap to the toolchain version of the module when you load the toolchain after the SYSTEM module?

Flamefire commented 2 weeks ago

But when would we need that? If there is a SYSTEM dependency in an EC the toolchain shouldn't have it too because SYSTEM modules are either toolchains or builddependencies of a toolchain. Or are there other cases?

I guess showing an error when the toolchain tries to load an already loaded module is actually the better behavior as a) I can't see where this is intended and b) would allow us to detect if and where we break existing behavior