HDF5 was not built with zlib while configuring netCDF-C

[gabriel2029]

Hello, I wanted to build the netCDF-C library from source and followed the tutorial from the official site, more precisely the sections "Building zlib from source", "Building hdf5 from source" and "Building netCDF-4 and the Remote Data Client from source". Based on these steps I created the following bash script:

#!/usr/bin/bash

# Definition of folders
ZDIR=/scratch/netcdf/zlib
H5DIR=/scratch/netcdf/hdf5
NCDIR=/scratch/netcdf/netcdf-c
NFDIR=/scratch/netcdf/netcdf-fortran
BUILDDIR=/scratch/netcdf/build

# Definition of versions
ZVER=1.2.11
H5VER=1.12.0
NCVER=4.7.4
NFVER=4.5.3

# Definition of compilers
export CC=/usr/bin/gcc
export CXX=/usr/bin/g++
export C++=/usr/bin/g++
export FC=/usr/bin/gfortran

# Build and install zlib
mkdir -p ${BUILDDIR}
cd ${BUILDDIR}
wget http://www.zlib.net/zlib-${ZVER}.tar.gz
tar -zxvf zlib-${ZVER}.tar.gz
rm zlib-${ZVER}.tar.gz
cd zlib-${ZVER}
./configure --prefix=${ZDIR}
make check #-j 4
make install

# Build and install hdf5
cd ${BUILDDIR}
wget https://hdf-wordpress-1.s3.amazonaws.com/wp-content/uploads/manual/HDF5/HDF5_${H5VER//./_}/source/hdf5-${H5VER}.tar.gz
tar -zxvf hdf5-${H5VER}.tar.gz
rm hdf5-${H5VER}.tar.gz
cd hdf5-${H5VER}
./configure --with-zlib=${ZDIR} --prefix=${H5DIR} --enable-hl
make check #-j 4
make install

# Build and install netcdf-c
cd ${BUILDDIR}
wget -O nc-${NCVER}.tar.gz https://github.com/Unidata/netcdf-c/archive/v${NCVER}.tar.gz
tar -zxvf nc-${NCVER}.tar.gz
rm nc-${NCVER}.tar.gz
cd netcdf-c-${NCVER}
CPPFLAGS='-I${H5DIR}/include -I${ZDIR}/include' LDFLAGS='-L${H5DIR}/lib -L${ZDIR}/lib' ./configure --prefix=${NCDIR}
make check #-j 4
make install

## Build and install netcdf-fortran
## define environment variables
#export LD_LIBRARY_PATH=${NCDIR}/lib:${LD_LIBRARY_PATH}
#
## go into builddir
#cd ${BUILDDIR}
#wget -O nf-${NFVER}.tar.gz https://github.com/Unidata/netcdf-fortran/archive/v${NFVER}.tar.gz
#tar -zxvf nf-${NFVER}.tar.gz
#rm nf-${NFVER}.tar.gz
#cd netcdf-fortran-${NCVER}
#CPPFLAGS='-I${NCDIR}/include LDFLAGS=-L${NCDIR}/lib' ./configure --prefix=${NFDIR}
#make check #-j 4
#make install

# Go back to builddir
cd ${BUILDDIR}

The building of zlib and HDF5 works fine, but the ./configure-command in the installation of netCDF-C fails due to the following error:

configure: error: HDF5 was not built with zlib, which is required. Rebuild HDF5 with zlib.
make: * No rule to make target 'check'.  Stop.
make: * No rule to make target 'install'.  Stop.

I cannot explain this, because hdf5 seems to be explicitly compiled with zlib. What did I miss in my script?

Operating system is CentOS 8 with GNU GCC 8.3.1 and GNU Make 4.2.1.

Best regards, Gabriel

[edwardhartnett]

When configure runs it produces an output file called config.log.

Can you attach that output file to this issue? Or look in it for the relevant section where this failure occurs and post that?

[gabriel2029]

Here is the config.log produced by netcdf-c: config.log Do you also need the config.log from HDF5?

[edwardhartnett]

OK, this is a bug that has been fixed for the next release.

Go into the directory where the HDF5 header files are, and create an empty file with the name H5pubconf.h.

Then try to run configure again and see if it works.

[gabriel2029]

Unfortunately, in the include directory of HDF5 there exists already a file named H5pubconf.h:

# pwd
/scratch/netcdf/hdf5/include
# ls
H5ACpublic.h  H5Epublic.h   H5FDmpio.h     H5Gpublic.h   H5Opublic.h   H5Rpublic.h               H5VLpassthru.h
H5api_adpt.h  H5ESpublic.h  H5FDmulti.h    H5IMpublic.h  H5overflow.h  H5Spublic.h               H5VLpublic.h
H5Apublic.h   H5FDcore.h    H5FDpublic.h   H5Ipublic.h   H5PLextern.h  H5TBpublic.h              H5Zpublic.h
H5Cpublic.h   H5FDdirect.h  H5FDros3.h     H5LDpublic.h  H5PLpublic.h  H5Tpublic.h               hdf5.h
H5DOpublic.h  H5FDfamily.h  H5FDsec2.h     H5Lpublic.h   H5Ppublic.h   H5version.h               hdf5_hl.h
H5Dpublic.h   H5FDhdfs.h    H5FDstdio.h    H5LTpublic.h  H5PTpublic.h  H5VLconnector.h
H5DSpublic.h  H5FDlog.h     H5FDwindows.h  H5MMpublic.h  H5pubconf.h   H5VLconnector_passthru.h
H5Epubgen.h   H5FDmpi.h     H5Fpublic.h    H5Mpublic.h   H5public.h    H5VLnative.h

(I compiled all libraries with make -j 4, because make check took very long.)

[edwardhartnett]

Then WTF can't if compile it? Stand by...

[edwardhartnett]

You are setting CPPFLAGS like this:

ac_cv_env_CPPFLAGS_value='-I${H5DIR}/include -I${ZDIR}/include'

Do you have H5DIR and ZDIR defined?

[gabriel2029]

I am running the script above, and H5DIR and ZDIR should be defined.

// Edit: Also chancing the compilers to those of Intel's parallelstudio caused the same error.

// Edit2: Building netcdf-c-4.6.3 seems to work without any issues.

[gus-correa]

I had the same issue. hdf5 is built with zlib, and yet configures gives me the same error reported by gabriel2029. Any suggestions/fixes?

Thank you, Gus Correa

[gabriel2029]

Hello Gus,

I solved this problem at first by using an older netCDF-C-version, after that by switching to MPI and some further parameter adjustments. My last working standing should be the following, however, I haven't looked for it quite a while: install_netcdf_gnu.sh.txt, install_netcdf_intel.sh.txt Especially the links used in the script could be outdated.

An easier way could be directly the usage of the spack package manager https://spack.io/, which enables you to build the most common packages with the compiler of your choice. But here you have to invest time in understanding the setup process.

Best regards, Gabriel

[gus-correa]

Thank you Gabriel.

Somehow netCDF-C 4.8.0 builds without a problem (but using it to build netCDF-fortran 4.5.3 still didn't work, trying more ...) It is the Fortran-90 API that I am trying to get working. The one from the Linux package is throwing an error in nf90_def_var_fill during compilation that sounds very much as a bug (mindset that the last parameter, the actual _FillValue must be a 4-byte integer).

But netCDF-C 4.7.4 doesn't build. I am surprised because the autotools should work smoothly, since everything is built with Gnu compilers (including hdf5, etc which are native Linux packages).

Not so long ago netCDF would bundle the C, Fortran and C++ APIs in a single and self-consistent tarball, with a release number, and everything would build smoothly, user-friendly. I miss that.

Gus Correa

On Thu, Jul 8, 2021 at 6:17 PM gabriel2029 @.***> wrote:

Hello Gus,

I solved this problem at first by using an older netCDF-C-version, after that by switching to MPI and some further parameter adjustments. My last working standing should be the following, however, I haven't looked for it quite a while: install_netcdf_gnu.sh.txt https://github.com/Unidata/netcdf-c/files/6787513/install_netcdf_gnu.sh.txt, install_netcdf_intel.sh.txt https://github.com/Unidata/netcdf-c/files/6787514/install_netcdf_intel.sh.txt Especially the links used in the script could be outdated.

An easier way could be directly the usage of the spack package manager https://spack.io/, which enables you to build the most common packages with the compiler of your choice. But here you have to invest time in understanding the setup process.

Best regards, Gabriel

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Unidata/netcdf-c/issues/1844#issuecomment-876780844, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHIZALWO2EOLUMZL233TPLLTWYPWJANCNFSM4RTXOWFQ .

[heralex]

For those of you using cmake, I had the same issue building netcdf-c-4.8.1 with hdf5-1.12.1. Changing CHECK_C_SOURCE_COMPILES("#include <H5public.h> in CMakeLists.txt, line 834 to CHECK_C_SOURCE_COMPILES("#include <H5pubconf.h> solved the issue for me. However, as gus-correa reported, building netcdf-c-4.8.0 works fine without the change above, so I guess this is merely a hack rather than a proper fix.

[edwardhartnett]

I believe this issue can be closed.