search

easybuilders / easybuild-easyblocks

Collection of easyblocks that implement support for building and installing software with EasyBuild.
https://easybuild.io
GNU General Public License v2.0
106 stars 285 forks source link

use relative paths to object files when compiling shared libraries in the OpenFOAM easyblock #3388

Closed bedroge closed 3 months ago

bedroge commented 4 months ago

In EESSI we ran into the following error when compiling OpenFOAM v2312 for Intel Skylake (see https://github.com/EESSI/software-layer/pull/563#issuecomment-2180005389):

make: /bin/sh: Argument list too long
make: *** [/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/wmake/makefiles/general:210: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so] Error 127

This worked fine for other CPU targets, but due to the CPU target names that we use, Skylake has the longest installation path ("intel/skylake_avx512"). Apparently, this makes the entire command for building the shared library libOpenFOAM.so just a bit too long. This command includes a large number (~600) of absolute paths to object files:

OMPI_CXX="g++" mpicxx -m64 -pthread -DOPENFOAM=2312 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof -Wno-attributes -Wno-unknown-pragmas -O2 -fno-tree-vectorize -march=native -fno-math-errno -std=c++14 -fuse-ld=bfd  -DNoRepository -ftemplate-depth-100  -I/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/build/linux64GccDPInt32Opt/src/OpenFOAM -DHAVE_LIBZ -DHAVE_EXTRAE -iquote. -IlnInclude -I/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/src/OpenFOAM/lnInclude -I/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/src/OSspecific/POSIX/lnInclude   -fPIC -shared -Xlinker --add-needed -Xlinker --no-as-needed  
/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/build/linux64GccDPInt32Opt/src/OpenFOAM/global/foamConfig.o
/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/build/linux64GccDPInt32Opt/src/OpenFOAM/global/globals.o
<about 600 more of these /paths/to/some.o> 
-L/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib \
    /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/libOSspecific.o -lz -L/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/dummy -lPstream  -o /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so

This PR fixes the issue by replacing the absolute paths to all these object files by relative ones. It does that by replacing the $WM_PROJECT_DIR (main installation path of this OpenFOAM version) part of the path by the relative path (relative to the current working dir) to this main installation directory for every object file. This means that the ~600 paths will then look something like:

../../../build/linux64GccDPInt32Opt/src/Pstream/dummy/UPstream.o

And this greatly reduces the length of this entire command, which made it compile without issues for our Skylake installation path.

Since the same code seems to be part of both different OpenFOAM forks and in all the versions that I checked, I haven't restricted this to a specific fork/version.

bedroge commented 4 months ago

Actually, this makefile was introduced in 4.x for the .org fork. For older versions, the file does not exist, but in that case the change just won't be applied (as it checks if the file exists), which should be fine.

For the .com version, this file should be present for all versions.

bedroge commented 4 months ago

Successful build with the EESSI bot for skylake: https://github.com/EESSI/software-layer/pull/563#issuecomment-2217797794

bedroge commented 4 months ago

@boegelbot please test @ generoso CORE_CNT=16 EB_ARGS="--installpath=/tmp/$USER/pr3388 OpenFOAM-8-foss-2020a.eb OpenFOAM-9-foss-2021a.eb OpenFOAM-10-foss-2022a.eb OpenFOAM-11-foss-2023a.eb OpenFOAM-v2106-foss-2021a.eb OpenFOAM-v2112-foss-2021b.eb OpenFOAM-v2206-foss-2022a.eb OpenFOAM-v2306-foss-2022b.eb OpenFOAM-v2312-foss-2023a.eb"

boegelbot commented 4 months ago

@bedroge: Request for testing this PR well received on login1

PR test command 'EB_PR=3388 EB_ARGS="--installpath=/tmp/$USER/pr3388 OpenFOAM-8-foss-2020a.eb OpenFOAM-9-foss-2021a.eb OpenFOAM-10-foss-2022a.eb OpenFOAM-11-foss-2023a.eb OpenFOAM-v2106-foss-2021a.eb OpenFOAM-v2112-foss-2021b.eb OpenFOAM-v2206-foss-2022a.eb OpenFOAM-v2306-foss-2022b.eb OpenFOAM-v2312-foss-2023a.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks /opt/software/slurm/bin/sbatch --job-name test_PR_3388 --ntasks="16" ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

Test results coming soon (I hope)...

*- notification for comment with ID 2218652517 processed* *Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).*
boegelbot commented 4 months ago

Test report by @boegelbot

Overview of tested easyconfigs (in order)

Build succeeded for 9 out of 9 (9 easyconfigs in total) cnx1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8 See https://gist.github.com/boegelbot/b5e53db5955ea2e03c35574f917e7ad3 for a full test report.