search

easybuilders / easybuild-easyblocks

Collection of easyblocks that implement support for building and installing software with EasyBuild.
https://easybuild.io
GNU General Public License v2.0
106 stars 285 forks source link

Replace `run_cmd` with `run_shell_cmd` in custom easyblock for MetalWalls #3415

Closed Crivella closed 2 months ago

Crivella commented 2 months ago

Replaced run_cmd with run_shell_cmd for 5.0 release

boegel commented 2 months ago

@boegelbot please test @ jsc-zen3 EB_ARGS="MetalWalls-21.06.1-foss-2023a.eb" EB_BRANCH=5.0.x

boegelbot commented 2 months ago

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ "5.0.x" != 'develop' ]]; then EB_BRANCH="5.0.x" ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/"5.0.x" source init_env_easybuild_develop.sh; fi; EB_PR=3415 EB_ARGS="MetalWalls-21.06.1-foss-2023a.eb" EB_REPO=easybuild-easyblocks EB_BRANCH="5.0.x" /opt/software/slurm/bin/sbatch --job-name test_PR_3415 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

Test results coming soon (I hope)...

*- notification for comment with ID 2296121135 processed* *Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).*
boegelbot commented 2 months ago

Test report by @boegelbot

Overview of tested easyconfigs (in order)

Build succeeded for 0 out of 1 (1 easyconfigs in total) jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18 See https://gist.github.com/boegelbot/9e690383331296798b5f39d5b71c7803 for a full test report.

Crivella commented 2 months ago

I tested also on my system with --rpath and i get the same error, but libplumed.so and libplumedKernel.so are in the same directory. Also I don't think we had this failure when building MW for EESSI, which should have RPATH enabled

Having a closer look at the build process, it seems that the plumed commands is used only to generate links to the Plumed.inc which than is used by the Makefile to link plumed as the full path to the library plus -ldl but no reference to -lplumedKernel. This causes the secondary dependency of libplumed.so -> libplumedKernel.so not being picked up as NEEDED in the elf and consequently by RPATH.

Is there a workaround that is implemented in the EESSI build for this? I did try to add a check to see if rpath and plumed are being used and run patchelf to add:

which seems to solve this, but wanted to check if there if there is already a more general solution that could be applied here.

boegel commented 2 months ago

Hmm, from the top of my head I don't recall us implementing a workaround in EESSI for this, so I'm also a bit puzzled here...

The installation in EESSI doesn't show the problems being caught here then?

Crivella commented 2 months ago

Yeah it went through without any problem, also ran the ReFrame test on my WS from the EESSI installation, and we've also started testing it in the test-suite (still not merged https://github.com/EESSI/test-suite/pull/164)

Crivella commented 2 months ago

As suggested by @boegel in a discussion it seems one of the dependencies (in this case PLUMED) was not build with RPATH enabled on the build cluster. Tested on my WS and rebuildig PLUMED with --rpath leads to a successful build of MW without any easyblock modifications

boegel commented 2 months ago

It looks like our set up jsc-zen3 is not exactly ideal, as build done with EasyBuild 4.x and 5.x are mixed...

So indeed, PLUMED was installed with EasyBuild 4.x (without RPATH).

Should be easy to work around though, I'll just trigger a rebuild of PLUMED too...

boegel commented 2 months ago

@boegelbot please test @ jsc-zen3 EB_ARGS="PLUMED-2.9.0-foss-2023a.eb MetalWalls-21.06.1-foss-2023a.eb" EB_BRANCH=5.0.x

boegelbot commented 2 months ago

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ "5.0.x" != 'develop' ]]; then EB_BRANCH="5.0.x" ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/"5.0.x" source init_env_easybuild_develop.sh; fi; EB_PR=3415 EB_ARGS="PLUMED-2.9.0-foss-2023a.eb MetalWalls-21.06.1-foss-2023a.eb" EB_REPO=easybuild-easyblocks EB_BRANCH="5.0.x" /opt/software/slurm/bin/sbatch --job-name test_PR_3415 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

Test results coming soon (I hope)...

*- notification for comment with ID 2296788214 processed* *Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).*
boegelbot commented 2 months ago

Test report by @boegelbot

Overview of tested easyconfigs (in order)

Build succeeded for 2 out of 2 (2 easyconfigs in total) jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18 See https://gist.github.com/boegelbot/99d3657d0125a6b4cd29a862708aca61 for a full test report.