Open Plasdom opened 1 month ago
Hey @Plasdom ! For the first error: It looks like you have to reinstall fftw3 as a shared library. I think the second (DWARF) error is because the system binutils /usr/bin/ld
is probably very old and can't parse the object files (I'm not sure if it's a BOUT++ or netcdf object).
Thanks for the help. I've rebuilt fftw with --enable-shared
and removed the flag -DBUILD_SHARED_LIBS=Off
from the bout++ build command, and can build successfully. The tests are failing due to not being able to find netcdf functions:
/usr/bin/ld: warning: libnetcdf.so.7, needed by /usr/local/netcdf/lib/libnetcdf_c++4.so, may conflict with libnetcdf.so.15
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcFile::NcFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, netCDF::NcFile::FileMode)'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcGroup::addVar(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, netCDF::NcType const&, std::vector<netCDF::NcDim, std::allocator<netCDF::NcDim> > const&) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcGroup::getVars[abi:cxx11](netCDF::NcGroup::Location) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcVar::putAtt(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, netCDF::NcType const&, double) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcType::getName[abi:cxx11]() const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcGroup::addGroup(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcVar::putAtt(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, netCDF::NcType const&, int) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcGroup::getGroup(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, netCDF::NcGroup::GroupLocation) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcGroup::getGroups[abi:cxx11](netCDF::NcGroup::GroupLocation) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcGroup::getDim(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, netCDF::NcGroup::Location) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcAtt::getValues(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcVar::getAtts[abi:cxx11]() const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcGroup::addDim(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcVar::putAtt(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcGroup::addDim(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long) const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcDim::getName[abi:cxx11]() const'
../../lib/libbout++.so.5.1.0: undefined reference to `netCDF::NcGroup::getVar(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, netCDF::NcGroup::Location) const'
collect2: error: ld returned 1 exit status
gmake[3]: *** [examples/conduction/conduction] Error 1
gmake[2]: *** [examples/conduction/CMakeFiles/conduction.dir/all] Error 2
gmake[1]: *** [examples/conduction/CMakeFiles/conduction.dir/rule] Error 2
gmake: *** [conduction] Error 2
The warning on the first line is probably a clue. The version of netcdf I'm using is 4.7.0. Looks like the version of binutils is 2.27 which is quite old, so do I need to update that?
Update: I have built the latest version of binutils and get the same error. The warning on the first line above was caused by using a different version of gcc from the one it seems netcdf was compiled with; fixing this removes the warning but does not fix the errors (undefined reference to ...
)
Can you post the content of examples/conduction/CMakeFiles/conduction.dir/link.txt
?
Have you tried removing the build folder, after you fixed the netcdf warning?
Yes, examples/conduction/CMakeFiles/conduction.dir/link.txt
looks like:
This is after removing the build folder and rebuilding.
On 8/5/24 15:32, Dominic Power wrote:
Yes, |examples/conduction/CMakeFiles/conduction.dir/link.txt| looks like: image.png (view on web) https://github.com/user-attachments/assets/f4f73baf-f171-4ccb-8fe7-503bd0e36d36
Can you please share the text rather then a picture?
/usr/local/gcc-7.4.0/bin/g++ -O2 -g -DNDEBUG -Wl,-rpath -Wl,/home/power8/installs/mpich-4.2.2/lib -Wl,--enable-new-dtags CMakeFiles/conduction.dir/conduction.cxx.o -o conduction -Wl,-rpath,/home/power8/bout/build-bout/lib:/home/power8/installs/mpich-4.2.2/lib:/usr/local/netcdf/lib:/home/power8/installs/fftw/lib ../../lib/libbout++.so.5.1.0 ../../lib/libfmt.so.10.1.1 ../../lib/libpvode.so.1.0.0 ../../lib/libpvpre.so.1.0.0 /home/power8/installs/mpich-4.2.2/lib/libmpicxx.so /home/power8/installs/mpich-4.2.2/lib/libmpi.so /usr/local/netcdf/lib/libnetcdf_c++4.so /usr/lib64/libnetcdf.so /home/power8/installs/fftw/lib/libfftw3.so /usr/lib64/libuuid.so -ldl
what does
nm -CD /usr/local/netcdf/lib/libnetcdf_c++4.so | grep 'netCDF::NcFile::NcFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, netCDF::NcFile::FileMode)'
say?
Is /usr/local/netcdf/lib/libnetcdf_c++4.so
really the netcdf-c++ library you want to use?
/usr/lib64/libnetcdf.so
also looks wrong, as you mentioned you have loaded a netcdf module.
You probably need to tell cmake to use the ones you want to use.
if the above has no output, what does:
nm -CD /usr/local/netcdf/lib/libnetcdf_c++4.so | grep 'netCDF::NcFile::NcFile'
give?
The first command has no output. The second has the following:
0000000000026ef0 T netCDF::NcFile::NcFile(std::string const&, netCDF::NcFile::FileMode)
0000000000027080 T netCDF::NcFile::NcFile(std::string const&, netCDF::NcFile::FileMode, netCDF::NcFile::FileFormat)
0000000000026dc0 T netCDF::NcFile::NcFile()
0000000000026ef0 T netCDF::NcFile::NcFile(std::string const&, netCDF::NcFile::FileMode)
0000000000027080 T netCDF::NcFile::NcFile(std::string const&, netCDF::NcFile::FileMode, netCDF::NcFile::FileFormat)
0000000000026dc0 T netCDF::NcFile::NcFile()
Is
/usr/local/netcdf/lib/libnetcdf_c++4.so
really the netcdf-c++ library you want to use?/usr/lib64/libnetcdf.so
also looks wrong, as you mentioned you have loaded a netcdf module.
Any thoughts on this?
As far as I can tell, the issue is that netcdf-c++ has been compiled against an old c++-standard, while you are using a newer c++ standard to compile bout++. As such you need to link against a netcdf-c++ that is also compiled with a newer c++ standard.
See previous comments as to what could be the cause of that.
I thought it might be something like that. I've tried building netcdf from source with no success so far, so I'll focus on that for now. Thanks for your help.
Have you tried building BOUT++ with:
cmake . -B build -DBOUT_DOWNLOAD_NETCDF_CXX4=ON
This will build the netcdf C++ library as part of the BOUT++ build.
It also looks like you're using the system gcc (7.4) rather than the modules one (12.2). I suspect you're also picking up the system netcdf too. Please can you run the following:
g++ --version
nc-config --all
mpirun --version
mpic++ -show
It could be that you've built mpich using the system compiler, so cmake is going to pick that up instead of the module one. If so, you'll either have to recompile mpich or use an MPI implementation from the module system
Have you tried building BOUT++ with:
cmake . -B build -DBOUT_DOWNLOAD_NETCDF_CXX4=ON
Yep, as you spotted BOUT++ is picking up the system netcdf and proceeding without building it, even with that flag set. I suspect building netcdf myself and using that with BOUT++ would fix the problem, but I'm hitting an error there which I haven't resolved yet (ddispatch.c:28:10: fatal error: /usr/local/include/curl/curl.h: Permission denied
).
Output from g++ --version
:
g++ (GCC) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Output from nc-config --all
:
This netCDF 4.3.3.1 has been built with the following features:
--cc -> gcc
--cflags -> -I/usr/include -I/usr/include/hdf
--libs ->
--has-c++ -> no
--cxx ->
--has-c++4 -> yes
--cxx4 -> g++
--fc -> gfortran
--fflags -> -I/usr/include
--flibs -> -lnetcdff
--has-f90 -> yes
--has-dap -> yes
--has-nc2 -> yes
--has-nc4 -> yes
--has-hdf5 -> yes
--has-hdf4 -> yes
--has-pnetcdf-> no
--prefix -> /usr
--includedir-> /usr/include
--version -> netCDF 4.3.3.1
Output from mpirun --version
:
HYDRA build details:
Version: 4.2.2
Release Date: Wed Jul 3 09:16:22 AM CDT 2024
CC: /usr/local/gcc-12.2/bin/gcc
Configure options: '--disable-option-checking' '--prefix=/home/power8/installs/mpich-4.2.2' '--with-hwloc=embedded' 'CC=/usr/local/gcc-12.2/bin/gcc' 'CXX=/usr/local/gcc-12.2/bin/g++' 'FC=/usr/local/gcc-12.2/bin/gfortran' '--cache-file=/dev/null' '--srcdir=.' 'CFLAGS= -O2' 'LDFLAGS=' 'LIBS=' 'CPPFLAGS= -DNETMOD_INLINE=__netmod_inline_ofi__ -I/home/power8/mpich-4.2.2/src/mpl/include -I/home/power8/mpich-4.2.2/modules/json-c -I/home/power8/mpich-4.2.2/modules/hwloc/include -D_REENTRANT -I/home/power8/mpich-4.2.2/src/mpi/romio/include -I/home/power8/mpich-4.2.2/src/pmi/include -I/home/power8/mpich-4.2.2/modules/yaksa/src/frontend/include -I/home/power8/mpich-4.2.2/modules/libfabric/include'
Process Manager: pmi
Launchers available: ssh rsh fork slurm ll lsf sge manual persist
Topology libraries available: hwloc
Resource management kernels available: user slurm ll lsf sge pbs cobalt
Demux engines available: poll select
Output from mpic++ -show
:
/usr/local/gcc-12.2/bin/g++ -I/home/power8/installs/mpich-4.2.2/include -L/home/power8/installs/mpich-4.2.2/lib -lmpicxx -Wl,-rpath -Wl,/home/power8/installs/mpich-4.2.2/lib -Wl,--enable-new-dtags -lmpi
CMake is sometimes weird about finding the wrong compiler. I found I often need something like:
cmake -DCMAKE_CXX_COMPILER=$(which g++) \
-DCMAKE_C_COMPILER=$(which gcc) \
-DMPI_CXX_COMPILER=$(which mpic++) \
-DMPI_C_COMPILER=$(which mpicc)
maybe something like that helps cmake find the right compiler + mpiwrappers
Setting the serial compiler is sometimes needed, as otherwise cmake things the flags do not work, because it tests them with the wrong compiler.
I'm trying to build the latest version of bout++ on LLNL's singe cluster (Red Hat Enterprise Linux v7.9). I am using the singe modules gcc (v12.2) and netcdf (v4.3.3.1), and versions of cmake (v3.30.1), mpich (v4.2.2) and fftw (v3.3) built from source. I can succesfully configure but not build with the following:
The error from cmake is many lines similar to:
I've also tried running cmake with the flag
-DBUILD_SHARED_LIBS=Off
. This allows bout++ to build successfully, but it then fails when building examples:The cache file is here: CMakeCache.txt