search

NCAR / ParallelIO

A high-level Parallel I/O Library for structured grid applications
Apache License 2.0
136 stars 53 forks source link

vpath build fails #1862

Open rjdave opened 3 years ago

rjdave commented 3 years ago

I am attempting to build PIO 2.5.3 on Linux with the Intel 17 compilers using MVAPICH2-2.2. I cloned the github repository, checked out the 2.5.3 tag, and ran autoconf -i. I am attempting a vpath (out of source) build using the configure script. here is a (severely) slimmed down version of the configure command I'm using:

../../pio/configure CC=mpicc FC=mpifort --enable-fortran --disable-shared --enable-netcdf-integration

the configure completes so I execute make and it errors here:

libtool: compile:  mpifort -DHAVE_CONFIG_H -I. -I../../../../pio/tests/unit -I../.. -I../../../../pio/src/flib -I/projects/dmcs_1/sw/packages/intel-17.0.4/mvapich2-2.2/netcdf/4.7.4/include -I/projects/dmcs_1/sw/packages/intel-17.0.4/mvapich2-2.2/phdf5/1.10.6/include -I/projects/dmcs_1/sw/packages/intel-17.0.4/mvapich2-2.2/pnetcdf/1.12.2/include -O3 -c ../../../../pio/tests/unit/global_vars.F90 -o global_vars.o
../../../../pio/tests/unit/global_vars.F90(9): error #7002: Error in opening the compiled module file.  Check INCLUDE paths.   [PIO]
  use pio
------^
../../../../pio/tests/unit/global_vars.F90(15): error #6592: This symbol must be a defined parameter, an enumerator, or an argument of an inquiry function that evaluates to a compile-time constant.   [PIO_MAX_NAME]
  integer, parameter :: str_len = pio_max_name, ntest=4
----------------------------------^
../../../../pio/tests/unit/global_vars.F90(27): error #6457: This derived type name has not been declared.   [IOSYSTEM_DESC_T]
  type(iosystem_desc_t), save :: pio_iosystem
-------^
../../../../pio/tests/unit/global_vars.F90(28): error #6457: This derived type name has not been declared.   [FILE_DESC_T]
  type(file_desc_t), save     :: pio_file
-------^
../../../../pio/tests/unit/global_vars.F90(36): error #6592: This symbol must be a defined parameter, an enumerator, or an argument of an inquiry function that evaluates to a compile-time constant.   [PIO_IOTYPE_NETCDF]
  integer, dimension(ntest) :: iotypes =  (/ PIO_iotype_netcdf,         &
---------------------------------------------^
../../../../pio/tests/unit/global_vars.F90(36): error #6973: This is not a valid initialization expression.
  integer, dimension(ntest) :: iotypes =  (/ PIO_iotype_netcdf,         &
---------------------------------------------^
../../../../pio/tests/unit/global_vars.F90(15): error #6404: This name does not have a type, and must have an explicit type.   [PIO_MAX_NAME]
  integer, parameter :: str_len = pio_max_name, ntest=4
----------------------------------^
../../../../pio/tests/unit/global_vars.F90(36): error #6404: This name does not have a type, and must have an explicit type.   [PIO_IOTYPE_NETCDF]
  integer, dimension(ntest) :: iotypes =  (/ PIO_iotype_netcdf,         &
---------------------------------------------^
../../../../pio/tests/unit/global_vars.F90(51): error #6404: This name does not have a type, and must have an explicit type.   [PIO_IOTYPE_NETCDF4P]
           (iotype.eq.PIO_iotype_netcdf4p) .or. &
----------------------^
../../../../pio/tests/unit/global_vars.F90(52): error #6404: This name does not have a type, and must have an explicit type.   [PIO_IOTYPE_NETCDF4C]
           (iotype.eq.PIO_iotype_netcdf4c) .or. &
----------------------^
../../../../pio/tests/unit/global_vars.F90(53): error #6404: This name does not have a type, and must have an explicit type.   [PIO_IOTYPE_PNETCDF]
           (iotype.eq.PIO_iotype_pnetcdf)
----------------------^
compilation aborted for ../../../../pio/tests/unit/global_vars.F90 (code 1)
gmake[3]: *** [global_vars.lo] Error 1
gmake[3]: Leaving directory `/projectsp/dmcs_1/src/vbuilds-pio/2.5.3-intel17mvapich2/tests/unit'
gmake[2]: *** [all-recursive] Error 1
gmake[2]: Leaving directory `/projectsp/dmcs_1/src/vbuilds-pio/2.5.3-intel17mvapich2/tests'
gmake[1]: *** [all-recursive] Error 1
gmake[1]: Leaving directory `/projectsp/dmcs_1/src/vbuilds-pio/2.5.3-intel17mvapich2'
gmake: *** [all] Error 2

The issue is that the compiled pio.mod file is at ../../src/flib (inside the build directory) not ../../../../pio/src/flib (inside the source directory). The compile line should have an include for src/flib in the build directory.

I'm not yet sure if the includes need to be corrected or if an extra include needs to be added. I will let you know if I gather any more details. For now I will try cmake or an in-source build and see if that goes more smoothly.

Interestingly, the 2.5.3 gzipped tarball downloaded from github actually has compiled .mod files in the source code so the error changes to error #7013: This module file was not generated by any release of this compiler.

edwardhartnett commented 3 years ago

OK, thanks for finding this.

I thought I was doing a make distcheck in the CI, which should have caught this.

rjdave commented 3 years ago

An in source build does (mostly) work for me. I say mostly because test_darray_async_many fails but that might partially be due to the fact that I set CFLAGS, FFLAGS, and FCFLAGS to -O3 since the cmake build passed that test and appears to have been built with "-debug minimal -O0". If I disable that one test 'make check' completes successfully.

That said, I did find another issue with vbuilds. After adding "-I../../src/flib" to CPPFLAGS the build system gets further but fails when looking for tests/general/util/pio_tf_f90gen.pl. Again, this perl script is in the source code but never copied to the build path.

rjdave commented 3 years ago

FYI: I happen to be looking at the pio 2.5.4 release tarball yesterday and noticed that it has the compiled .mod files in src/flib again. I DID NOT attempt to do a vbuild to determine if the -I/path/to/src/flib and copying of tests/general/util/pio_tf_f90gen.pl is still necessary.

edwardhartnett commented 3 years ago

I will not have time to look at this until the end of the Summer at the earliest. I'm all booked up until then. ;-)

If you want to submit a PR with fixes, that would be most welcome...

edwardhartnett commented 3 years ago

OK, this is the CMake build, correct?

You say:

it has the compiled .mod files in src/flib again.

Do you mean we fixed this and then it broke again?

Do you know what the fix for this should be?

jedwards4b commented 3 years ago

No this is not CMake - it's autoconf.

edwardhartnett commented 3 years ago

OK, I will check...

rjdave commented 3 years ago

This is with autotools. I have only compiled 2.5.3 and 2.5.4 so I don't know if this used to work or not

vpath autotools builds don't work regardless of whether the source code is obtained with a git clone or release tarball because:

  1. The include for src/flib is pointing to the source code (../../../../pio/src/flib in my case) so there is either no .mod files to include (git clone) or the mod files that are there are built with the wrong compiler (release tarball).
  2. tests/general/util/pio_tf_f90gen.pl is not copied from the source code to the build tree by the the configure script

In-source autotools builds from a release tarball source code fail because:

  1. src/flib already has compiled .mod files

However, if make clean is issued before make the problem goes away because make clean removes the compiled .mod files in src/flib.

In-source autotools builds from a git clone obtained source code do not have an issue since:

  1. The include path (src/flib) is correct
  2. No .mod files exist yet in src/flib
  3. tests/general/util/pio_tf_f90gen.pl exists

The already compiled .mod files problem appears to be confined to the src/flib directory as the release tarball does not contain .mod or .o files in any other directories.

The way I successfully do a vpath build is to run configure with -I../../src/flib added to CPPFLAGS. Then, after configure completes, I copy tests/general/util/pio_tf_f90gen.pl from the source code into the same directory in my build tree. Then I run make, remove "test_darray_async_many" from tests/cunit/run_tests.sh ( #1867), make check, and make install.