search

easybuilders / easybuild

EasyBuild - building software with ease
http://easybuild.io
GNU General Public License v2.0
464 stars 143 forks source link

Error when upgrade to 4.4.0 #710

Closed sit-upf-edu closed 3 years ago

sit-upf-edu commented 3 years ago

Hi,

I was trying to upgrade EasyBuild to the last version from the 4.3.4 version, and I've got this error:

_eb --install-latest-eb-release
== Temporary log file in case of crash /xxx/xxx/tmp/easybuild-4qX20y.log

WARNING: Deprecated functionality, will no longer work in v5.0: Support for Lmod version < 7.0.0 is deprecated, found version 6.6.3; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information

== found valid index for /xxx/xxx/noarch/software/EasyBuild/4.3.4/easybuild/easyconfigs, so using it...

WARNING: Deprecated functionality, will no longer work in v5.0: Use of 'dummy' toolchain is deprecated, use 'system' toolchain instead; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information

== processing EasyBuild easyconfig /tmp/eb-8Eap_L/tmpYZczHg/easybuilders/easybuild-easyconfigs-develop/easybuild/easyconfigs/e/EasyBuild/EasyBuild-4.4.0.eb
== building and installing EasyBuild/4.4.0...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== restore after iterating...
== postprocessing...
== sanity checking...
== FAILED: Installation ended unsuccessfully (build directory: /xxx/xxx/noarch/build/EasyBuild/4.4.0/dummy-dummy): build failed (first 300 chars): Module command '/xxx/xxx/lmod/lmod/libexec/lmod python --terse list' failed with exit code 1; stderr: Lmod has detected the following error: MODULEPATH has changed: run "module
update" to repair._

Thanks in advance!

boegel commented 3 years ago

@sit-upf-edu Hmmm, not sure what is going on there... Any chance you can share a debug log file for this (collected when using eb --debug)?

Any other problems with the modules tool? It seems like it isn't happy with the $MODULEPATH value for some reason...

Do you see the same problem is you run module update first, before using eb?

sit-upf-edu commented 3 years ago

Yes, I've got the same error running "module update"

boegel commented 3 years ago

@sit-upf-edu Then you should try and figure out this problem first, since it's not directly related to EasyBuild...

Which modules tool are you using? What does module --version produce as output?

EasyBuild is detecting that you're using an old version of Lmod, so I strongly recommend upgrading to a more recent version...

i-mtz commented 3 years ago

We are having issues too upgrading to 4.4.0, I tried from 4.3.4, 4.3.3 and 3.9.4 to discard any issue with previous release, but the error is the same.

[easybuild-EasyBuild-4.4.0-20210607.105341.KLtti.log](https://github.com/easybuilders/easybuild/files/6607391/easybuild-EasyBuild-4.4.0-20210607.105341.KLtti.log)
$ eb --install-latest-eb-release
== Temporary log file in case of crash /scratch/eb-sko_Lo/easybuild-dgGQ89.log
== found valid index for /scicore/soft/apps/EasyBuild/4.3.4/easybuild/easyconfigs, so using it...

WARNING: Deprecated functionality, will no longer work in v5.0: Use of 'dummy' toolchain is deprecated, use 'system' toolchain instead; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information

== processing EasyBuild easyconfig /scratch/eb-sko_Lo/tmp2QY0lk/easybuilders/easybuild-easyconfigs-develop/easybuild/easyconfigs/e/EasyBuild/EasyBuild-4.4.0.eb
== building and installing EasyBuild/4.4.0...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== restore after iterating...
== postprocessing...
== sanity checking...
== FAILED: Installation ended unsuccessfully (build directory: /scicore/home/admin/soft/.local/easybuild/build/EasyBuild/4.4.0/dummy-dummy): build failed (first 300 chars): Module command '/export/soft/lua_lmod/centos7/lmod/lmod/libexec/lmod python load EasyBuild/4.4.0' failed with exit code 1; stderr: Lmod has detected the following error: The sciCORE cluster prevents the automatic swapping of modules with same name. It's recommended
that you run 'module purge' before (took 23 sec)
== Results of the build can be found in the log file(s) /scratch/eb-sko_Lo/easybuild-EasyBuild-4.4.0-20210607.104254.ulAYu.log
ERROR: Build of /scratch/eb-sko_Lo/tmp2QY0lk/easybuilders/easybuild-easyconfigs-develop/easybuild/easyconfigs/e/EasyBuild/EasyBuild-4.4.0.eb failed (err: "build failed (first 300 chars): Module command '/export/soft/lua_lmod/centos7/lmod/lmod/libexec/lmod python load EasyBuild/4.4.0' failed with exit code 1; stderr: Lmod has detected the following error: The sciCORE cluster prevents the automatic swapping of modules with same name. It's recommended\nthat you run 'module purge' before")

This is on a CentOS 7.7 node, with LMOD 7.8.11.

The --install-latest-eb-release method to update EasyBuild has been working fine before the 4.4.0 release.

boegel commented 3 years ago

@i-mtz The The sciCORE cluster prevents the automatic swapping of modules with same name message strongly suggests this is happening because of a local Lmod configuration?

The problem seems to be that you have EasyBuild loaded as a module, and then the module load EaysBuild that is done during the sanity check is tripping a check in your Lmod configuration since loading a module with the same name twice is disallowed?

@pescobar Do you know more about this?

pescobar commented 3 years ago

@boegel you are right, the Lmod setup in our cluster prevents automatic module swap. The point is that the Lmod setup in our cluster has been like this for years and we always upgraded like this:

module load EasyBuild
eb --install-latest-eb-release

This problem only appeared when trying to upgrade to latest 4.4.0. Maybe before 4.4.0 the sanity check step was doing module unload && module load but this is no longer the case in 4.4.0?

pescobar commented 3 years ago

btw, I guess we can workaround the issue upgrading like this:

module load EasyBuild
export LMOD_DISABLE_SAME_NAME_AUTOSWAP="no"
eb --install-latest-eb-release

I asked @i-mtz to report the issue just in case this change is breaking the workflow for anyone else

i-mtz commented 3 years ago

Indeed running the upgrade process with LMOD_DISABLE_SAME_NAME_AUTOSWAP="no" works. As updating to the newer releases was working with our current setup we decided to mention it, as @pescobar pointed out, in case this could impact other users.

sit-upf-edu commented 3 years ago

Updating lmod it was fixed. Thanks

surak commented 2 years ago

| LMOD_DISABLE_SAME_NAME_AUTOSWAP="no" works.

This doesn't work for me. Will have to find a manageable way to upgrade lmod on ubuntu.