search

Parallel-NetCDF / PnetCDF

Source code repository of PnetCDF library and utilities
https://parallel-netcdf.github.io
Other
82 stars 23 forks source link

Nc4 #50

Closed khou2020 closed 5 years ago

khou2020 commented 5 years ago

allow linking to netcdf built with pnetcdf support

wkliao commented 5 years ago

This has been fixed in 6f2c74a6f2ec0f56aee457ee9723b5870a0baf8e please check and close this PR

khou2020 commented 5 years ago

6f2c74a only returns error from NetCDF. I added a debug trance so we can know where is the error first known in pnetcdf.

This pr also allow linking against netcdf with pnetcdf support where it is blocked in the master branch.

wkliao commented 5 years ago

What is the configure command used to allow to build PnetCDF with NetCDF4 that is built with --enable-pnetcdf? Did you tried both static and shared?

khou2020 commented 5 years ago

I removed the section that detects pnetcdf support and allow configure to continue. The error is now a warning. It is tested in static link but not shared.

wkliao commented 5 years ago

This issue has been reported #33. Run "make check" and see. The other way around, i.e. building NetCDF4 with PnetCDF built with --enable-netcdf4, also failed.

khou2020 commented 5 years ago

I didn't encounter this error while I run make check. fortran test also passed. It may only apply to shared link or it's a special case in their system.

wkliao commented 5 years ago

Try the followings. (all static builds)

  1. build PnetCDF 1.11.1 (no shared library)
  2. build NetCDF with --disable-shared and --with-pnetcdf using 1.
  3. build PnetCDF master branch using 2.
khou2020 commented 5 years ago

I already tried it on my desktop (ubuntu) before made the change. I built the master branch of pnetcdf without nc4. Then, I build netcdf using the pnetcdf i built. Later I build pnetcdf with nc4 using that netcdf. I am using netcdf 4.6.3.

wkliao commented 5 years ago

If that passed in your case, you must be using the same folder for all installations.

khou2020 commented 5 years ago

They are in different folders. Only hdf5 and original pnetcdf (for netcdf) are in the same folder (/home/khl7265/.local).

netcdf: ./configure --disable-shared --enable-parallel-tests --prefix=/home/khl7265/.local/netcdf --enable-pnetcdf --with-pnetcdf=/home/khl7265/.local/ncmpi_nc4 --disable-dap --with-pnetcdf is not used by netcdf config.

~/.local/netcdf/bin/nc-config --libs -L/home/khl7265/.local/netcdf/lib -L/home/khl7265/.local/lib -lnetcdf -lpnetcdf -lhdf5_hl -lhdf5 -lm -ldl -lz

pnetcdf_nc4: ./configure --enable-debug --disable-shared --enable-netcdf4 --with-netcdf4=/home/khl7265/.local/netcdf MPICC=mpicc MPICXX=mpicxx MPIF77=mpif77 MPIF90=mpif90

wkliao commented 5 years ago

I think your build uses the PnetCDF library already installed under /use/local. You can verify by search "checking for full path of header file pnetcdf.h" in the file config.log. Also, the path is set in variable ac_cv_header_path_pnetcdf_h

khou2020 commented 5 years ago

I don't have pre build pnetcdf on my desktop. It is indeed using the one i built for testing.

configure:18559: checking for full path of header file pnetcdf.h configure:18575: mpicc -E -I/home/khl7265/.local/include conftest.c

ac_cv_header_path_pnetcdf_h=/home/khl7265/.local/include/pnetcdf.h

wkliao commented 5 years ago

How did your netcdf build find the installation path of your pnetcdf under /home/khl7265/.local? Your configure command for netcdf does not contain that path, and somehow the line below from your config.log shows

configure:18575: mpicc -E -I/home/khl7265/.local/include conftest.c

Check line 7 of your config.log for the full configure command used.

khou2020 commented 5 years ago

NetCDF does not take --with- option, it relies on user to set CPPFLAGS and LDFLAGS manually (does not appear in config,log). The same goes for hdf5. The option I posted earlier are form line 7 of config.log

Here's the command found in my terminal history. I added --with-pnetcdf at first but turns out it does not recognize such option.

CC=mpicc CPPFLAGS="-I/home/khl7265/.local/include" LDFLAGS="-L/home/khl7265/.local/lib" LDADD="-lnetcdf" ./configure --disable-shared --enable-parallel-tests --prefix=/home/khl7265/.local/netcdf --enable-pnetcdf --with-pnetcdf=/home/khl7265/.local/ncmpi_nc4 --disable-dap

wkliao commented 5 years ago

Your global environment variable CPPFLAGS is reused when building PnetCDF. PnetCDF users may not follow this way to build NetCDF and PnetCDF.

khou2020 commented 5 years ago

I did not set CPPFLAGS when building pnetcdf with nc4. Setting CPPFLAGS is the way suggested by netcdf document to enable pnetcdf support. I believe it is the only way user can do to enable pnetcdf support. https://www.unidata.ucar.edu/software/netcdf/docs/getting_and_building_netcdf.html

wkliao commented 5 years ago

Check CPPFLAGS in config.log of your PnetCDF build. It should not show there.

khou2020 commented 5 years ago

It is empty.

CONFIGURE_ARGS_CLEAN='--enable-debug --disable-shared --enable-netcdf4 --with-netcdf4=/home/khl7265/.local/netcdf MPICC=mpicc MPICXX=mpicxx MPIF77=mpif77 MPIF90=mpif90' CONFIG_DATE='Sat May 4 20:30:16 CDT 2019' CPP='/usr/bin/mpicc -E' CPPFLAGS='' CXX='/usr/bin/mpicxx ' CXXCPP='/usr/bin/mpicxx -E' CXXDEPMODE='depmode=gcc3' CXXFLAGS='-g -O0' CYGPATH_W='echo' DECL_MPI_OFFSET_FALSE='#'

wkliao commented 5 years ago

Here is the explanation. The PnetCDF library used to build your NetCDF is installed in /home/khl7265/.local/lib. Then your NetCDF library is built and installed in /home/khl7265/.local/netcdf/lib When building PnetCDF with --enable-netcdf4, the configure only knows /home/khl7265/.local/netcdf/lib, but not /home/khl7265/.local/lib

The reason your build knows /home/khl7265/.local/lib is because of nc-config command. But one some systems, such as Cori, nc-config is not installed.

It is not a good idea to allow to use a NetCDF that is already built with PnetCDF. There is no point to support this option. I will keep it disabled.

wkliao commented 5 years ago

not merged