Closed mboisson closed 1 year ago
According to this, it worked at least up to version 4.6.0:
for f in $(grep -rl "configopts.*pyshortver" /cvmfs/soft.computecanada.ca/easybuild/ebfiles_repo/2020/); do grep easybuild-framework_version $f; done
"easybuild-framework_version": "4.4.1-r73a32642a48430484a7c9a4654e7afda27268be4",
"easybuild-framework_version": "4.3.3-r15719a0ece2011cea9ce176d1763cdc620b188f4",
"easybuild-framework_version": "4.6.0-ra6dbb4b22832c431590f725219d89d82c9946dba",
"easybuild-framework_version": "4.6.0-ra6dbb4b22832c431590f725219d89d82c9946dba",
"easybuild-framework_version": "4.6.0-ra6dbb4b22832c431590f725219d89d82c9946dba",
"easybuild-framework_version": "4.6.0-ra6dbb4b22832c431590f725219d89d82c9946dba",
"easybuild-framework_version": "4.3.3-r3ff498d1f735c51b313c04c5e4abeeaecc23589b",
"easybuild-framework_version": "4.3.3-r3ff498d1f735c51b313c04c5e4abeeaecc23589b",
"easybuild-framework_version": "4.2.2-r64415f53d9bdff498788ce494f68a98641a3883d",
"easybuild-framework_version": "4.2.2-r64415f53d9bdff498788ce494f68a98641a3883d",
"easybuild-framework_version": "4.4.0-r5824ede87a67bf66b912bcb3f7d73c27835716af",
"easybuild-framework_version": "4.4.0-r5824ede87a67bf66b912bcb3f7d73c27835716af",
"easybuild-framework_version": "4.4.0-r5824ede87a67bf66b912bcb3f7d73c27835716af",
"easybuild-framework_version": "4.3.3-r15719a0ece2011cea9ce176d1763cdc620b188f4",
Looking at one of these recipes, built with EasyBuild 4.6.0:
grep "run.py" /cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0/easybuild/easybuild-HOOMD-blue-3.6.0-20221117.183917.log | grep cmake | grep running
== 2022-11-17 18:03:15,250 run.py:233 INFO running cmd: cmake -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER='mpicc' -DCMAKE_C_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_CXX_COMPILER='mpicxx' -DCMAKE_CXX_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_Fortran_COMPILER='mpifort' -DCMAKE_Fortran_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=FALSE -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DENABLE_XHOST=OFF -DCMAKE_SKIP_INSTALL_RPATH=ON -DENABLE_MPI=ON -DENABLE_CUDA=OFF -DENABLE_TBB=ON -DENABLE_LLVM=OFF -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0/lib/python3.8/site-packages /tmp/ebuser/avx2/HOOMDblue/3.6.0/gompi-2020a-mpi/hoomd-v3.6.0/
== 2022-11-17 18:15:51,643 run.py:233 INFO running cmd: cmake -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER='mpicc' -DCMAKE_C_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_CXX_COMPILER='mpicxx' -DCMAKE_CXX_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_Fortran_COMPILER='mpifort' -DCMAKE_Fortran_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=FALSE -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DENABLE_XHOST=OFF -DCMAKE_SKIP_INSTALL_RPATH=ON -DENABLE_MPI=ON -DENABLE_CUDA=OFF -DENABLE_TBB=ON -DENABLE_LLVM=OFF -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0/lib/python3.9/site-packages /tmp/ebuser/avx2/HOOMDblue/3.6.0/gompi-2020a-mpi/hoomd-v3.6.0/
== 2022-11-17 18:27:07,729 run.py:233 INFO running cmd: cmake -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER='mpicc' -DCMAKE_C_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_CXX_COMPILER='mpicxx' -DCMAKE_CXX_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_Fortran_COMPILER='mpifort' -DCMAKE_Fortran_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=FALSE -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DENABLE_XHOST=OFF -DCMAKE_SKIP_INSTALL_RPATH=ON -DENABLE_MPI=ON -DENABLE_CUDA=OFF -DENABLE_TBB=ON -DENABLE_LLVM=OFF -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0/lib/python3.10/site-packages /tmp/ebuser/avx2/HOOMDblue/3.6.0/gompi-2020a-mpi/hoomd-v3.6.0/
This commit from @Flamefire is my first suspect. It messes with configopts
and has no explicit code to handle templates
https://github.com/easybuilders/easybuild-easyblocks/commit/0b78b0280ebbd8978110ce3853e265e515425a40
If I've got my testing correct:
CMake
only - not ConfigureMake
If I've got my testing correct:
- This broke between 4.6.0 and 4.6.1
- This impacts
CMake
only - notConfigureMake
That would match the above commit, I think
I confirmed that this can not be reproduced using MEGAHIT-1.2.9-GCCcore-9.3.0.eb
(which uses multi_deps
+ CMakeMake
) when fiddling with preconfigopts
, so looks specific to CMakeMake
and configopts
.
This mean is indeed a likely culprit, so I'll move this issue into the easyblocks repo...
There's another recently opened issue (https://github.com/easybuilders/easybuild-easyconfigs/issues/17259) caused by this PR https://github.com/easybuilders/easybuild-easyblocks/pull/2514
To avoid running into this in the future, it may be worth displaying a warning when __setitem__
is called with enable_templating = True
in easyconfig.py
?
To avoid running into this in the future, it may be worth displaying a warning when
__setitem__
is called withenable_templating = True
ineasyconfig.py
?
Actually the issue is __getitem__
as __setitem__
doesn't do any templating. Furthermore we do a lot of modifications with enabled templating without running into this issue.
I haven't found any of our ECs to be affected by this. If anyone can share one to test this that would help.
Cmd used: grep -F configopts $(grep -F --files-with-matches multi_deps $(grep -rF --files-with-matches CMakeMake easybuild-easyconfigs/easybuild/easyconfigs)) | grep -F '%(py'
That PR may have another regression related to iterating options: First time the configopts
are set/changed they are as specified by the easyconfig. During the next iteration they are already changed.
Example (after fix) of a single run with 2 python versions:
configopts=-DPYVER=%(pyshortver)s
running command "cmake -DCMAKE_INSTALL_PREFIX=/beegfs/ws/1/s3248973-EasyBuild/easybuild-haswell/software/MetaBAT/2.14-gompi-2019a -DBoost_NO_SYSTEM_PATHS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DBOOST_ROOT=/sw/installed/Boost/1.70.0-gompi-2019a -DPYVER=3.7 /tmp/s3248973-EasyBuild/MetaBAT/2.14/gompi-2019a/MetaBAT-2.14/"
configopts=-DCMAKE_INSTALL_PREFIX=/tmp/easybuild-tmp/eb-D18oYP/__ROOT__/beegfs/ws/1/s3248973-EasyBuild/easybuild-haswell/software/MetaBAT/2.14-gompi-2019a -DBoost_NO_SYSTEM_PATHS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DBOOST_ROOT=/sw/installed/Boost/1.70.0-gompi-2019a -DPYVER=%(pyshortver)s
running command "cmake -DCMAKE_INSTALL_PREFIX=/beegfs/ws/1/s3248973-EasyBuild/easybuild-haswell/software/MetaBAT/2.14-gompi-2019a -DBoost_NO_SYSTEM_PATHS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DBOOST_ROOT=/sw/installed/Boost/1.70.0-gompi-2019a -DPYVER=2.7 /tmp/s3248973-EasyBuild/MetaBAT/2.14/gompi-2019a/MetaBAT-2.14/"
To avoid running into this in the future, it may be worth displaying a warning when
__setitem__
is called withenable_templating = True
ineasyconfig.py
?Actually the issue is
__getitem__
as__setitem__
doesn't do any templating. Furthermore we do a lot of modifications with enabled templating without running into this issue.
I disagree the issue is __getitem__
. While __getitem__
is what resolves the templates, __setitem__
is what wrongly replaces values with templates unresolved by values with templates resolved. My point is that one should probably never use __setitem__
with template values resolved.
It is reasonable to call __getitem__
without disabling templating (that's what it's for, obviously), but it is unreasonable to call __setitem__
with the value obtained from __getitem__
with templates resolved. The dangerous action here is __setitem__
I haven't found any of our ECs to be affected by this. If anyone can share one to test this that would help. Cmd used:
grep -F configopts $(grep -F --files-with-matches multi_deps $(grep -rF --files-with-matches CMakeMake easybuild-easyconfigs/easybuild/easyconfigs)) | grep -F '%(py'
You will find many in our fork https://github.com/ComputeCanada/easybuild-easyconfigs
At least, I would add a test like the one proposed in https://github.com/easybuilders/easybuild-framework/pull/4212
so that the build fails hard and clearly, rather than failing silently and letting the user to ponder
There's another recently opened issue (easybuilders/easybuild-easyconfigs#17259) caused by this PR #2514
That is actually a different issue. I fixed that with a patch in https://github.com/easybuilders/easybuild-easyconfigs/pull/17288
At least, I would add a test like the one proposed in easybuilders/easybuild-framework#4212
so that the build fails hard and clearly, rather than failing silently and letting the user to ponder
Not sure how we can reasonably do that as we do self.cfg[...] = ...
or even self.cfg[...] += ...
almost everywhere. And even self.cfg.update
(which we also use a lot) doesn't handle the templating although there are drawbacks doing that (deduplication may fail).
I am not sure exactly when it happened, but recent EasyBuild versions broke template replacement in
configopts
when usingmulti_deps
, i.e. :for a
CMakeMake
EasyBlock (maybe others, I have not tested) no longer yield builds with differentconfigopts
argument based on%(pyshortver)s
.In a previous build log,
%(pyshortver)s
gets replaced for every iteration:while in a recent build, it keeps the same value as for the first iteration: