Closed SeanBryan51 closed 5 months ago
Sounds like when we think of a release of CABLE, we may want to release benchcab
independently of hh5 environments... Still need to think on this one but this is annoying.
The issue is due to environment variables being set which affect the behaviour the build, notably LDFLAGS
and CMAKE_PREFIX_PATH
:
$ module load conda/analysis3-unstable
$ echo $LDFLAGS
-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.01/lib -Wl,-rpath-link,/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.01/lib -L/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.01/lib
$ echo $CMAKE_PREFIX_PATH
/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.01:/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.01/x86_64-conda-linux-gnu/sysroot/usr
A quick fix would be to unset these variables before invoking CMake.
@dsroberts I noticed there are other environment variables being set when loading conda/analysis3-unstable
which may impact build systems (e.g. CC
, CFLAGS
, CPPFLAGS
, ...). It seems strange that these variables are being exported to the user environment. Do you know where these variables are coming from?
Hi @SeanBryan51 Yep. These come from the environment activation script for gcc_linux-64
found here: /g/data/hh5/public/apps/miniconda3/envs/analysis3-24.01/etc/conda/activate.d/activate-gcc_linux-64.sh
. gcc_linux-64
is bought in by parcels
, which is a dependency of some COSIMA recipes, so it can't just be removed. The conda
module works by running conda activate
in a 'blank' environment and parsing the output of the env
command. There is some level of filtering, but I'm not sure we can assume that no one ever wants to build against the analysis3 environments. What you've done with passing env
to subprocess.run
is probably the most sensible solution, though I think rather than removing the LDFLAGS
and CMAKE_PREFIX_PATH
environment variables entirely, you could create copies of them with references to /g/data/hh5/...
removed, then pass those to subprocess.run
.
CABLE is linked against incorrect libraries for netcdf and MPI when running benchcab (v4.0.2) from the hh5 conda environment.
The following behaviour only occurs when running from the hh5 conda environment and not when running from the
benchcab-dev
environment.Steps to reproduce:
Running the above causes the build to fail when compiling the MPI executable:
Output from CMake shows that we are linking against an MPI library found in the conda environment:
when the MPI_Fortran path should instead be pointing to
/apps/openmpi/4.1.0/lib/...
.The serial executable compiles successfully but is linked against the netcdf-fortran library found in the conda environment:
when this should point to
/apps/netcdf/4.7.4/lib/...
.Running the serial executable crashes due to undefined symbols from the netcdf library:
The PBS job script outputs:
Inspecting the standard output from CABLE: