visit-dav / visit

VisIt - Visualization and Data Analysis for Mesh-based Scientific Data
https://visit.llnl.gov
BSD 3-Clause "New" or "Revised" License
446 stars 117 forks source link

libhdf5_hl* not being installed #19692

Closed jfavre closed 3 months ago

jfavre commented 4 months ago

I compile VisIt (3.4.1) in two steps 1) ./src/tools/dev/scripts/build_visit --cc /usr/bin/gcc-12 --cxx /usr/bin/g++-12 --no-visit --parallel --llvm --mesagl --vtk9 --system-python --qwt --qt --silo --hdf5 --system-cmake --icet --ospray --ispc --embree --tbb --mfem --conduit --xdmf

then copy my hostname.cmake to the ./src/config-site

2) apply cmake with different options make && make install

I get run-time failures on startup because the libhdf5_hl were not copied by the make install if I copy them manually (see below), I can finally run cp third_party/hdf5/1.8.14/linux-x86_64_gcc-12/lib/libhdf5_hl /3.4.1/linux-x86_64/lib

biagas commented 4 months ago

I don't think VisIt truly needs hdf5_hl, @markcmiller86 says we build it for MOAB, but you didn't request MOAB. And yes, you are 100% correct, we aren't installing them. Needs a modification to our FindHDF5.cmake for the non-windows path. Interesting that you've run into this, but we haven't.

I noticed you are running make install. I don't think that is completely supported for Linux builds, it may be missing some secret sauce. Our process is make package then run visit-install on that created tarball. Just an FYI if there are other issues running from your make installed version.

biagas commented 4 months ago

So I probably misspoke. Doing more investigating on visit's/thirdparty usage of hdf5_hl.

jfavre commented 4 months ago

the run-time error I get is:

/3.4.1/linux-x86_64/bin/engine_par: error while loading shared libraries: libhdf5_hl.so.9: cannot open shared object file: No such file or directory
biagas commented 4 months ago

Thanks. Its an easy fix which I will put in today. There are several of our thirdparty libraries that depend on the hdf5_hl library, but we only register NETCDF as having that dependency. So if NETCDF isn't built, then Visit isn't picking up that dependency and installing it. Flawed logic on our part. Thanks for the report!

biagas commented 4 months ago

Synopsis of the problem:

VisIt has an indirect dependence on hdf5_hl, but our Find HDF5 module doesn’t look for hdf5_hl, so it isn't being installed from there.

Why do we need hdf5_hl?

MOAB and NETCDF have direct dependence on hdf5_hl.

MFEM thinks it needs hdf5_hl because it may build with NETCDF. hdf5_hl is automatically searched for and added to its link line. Dependence that doesn't need to exist, but does, and that un-needed dependence is pulled into VisIt.

How does hdf5_hl get installed for our releases?

In our config-site files, NETCDF_LIBDEP lists hdf5_hl, so when VisIt searches for the NETCDF_LIBDEP, hdf5_hl gets installed.

MOAB_LIBDEP doesn’t list hdf5_hl.

MFEM_LIBDEP doesn't list hdf5_hl.

So if NETCDF isn't part of the build, but MOAB or MFEM are, the hdf5_hl dependence is missed and it is not installed.

Options for fix:

Modify FindHDF5.cmake to always look for hdf5_hl so it will always get installed. Problem: What if VisIt is built without MOAB, NETCDF, or MFEM? There is no dependency chain then, and we build and install against an unneeded library.

Could change MOAB config-site listing to add hdf5_hl in its LIBDEP. That should work in the same way that it works when NETCDF is enabled.

Could also do that for MFEM. Perhaps creating a patch for MFEM so that it doesn't search for or add hdf5_hl to its link line. would be better. Then MFEM has no dependence on it at all.
Seems fine for VisIt builds of MFEM since we never enable their NETCDF support.

@markcmiller86 since you're involved in a revamping of hdf5/moab builds, what are your thoughts?

cyrush commented 3 months ago

We will solve in FindHDF5 to avoid surprises with other libs.