search

NOAA-GSL / SENA-c_sw

Computational kernel, c_sw, extracted from FV3 for performance and design analysis.
Other
0 stars 4 forks source link

On ARM with flang #2

Open LyndStringer opened 3 years ago

LyndStringer commented 3 years ago

On the Sandia NL ARM machine "stria", I tried to get flang working with the c_sw kernel. The command:

export CMAKE_PREFIX_PATH=/opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5

tells CMAKE where the netcdf directory is located. CMAKE then tries to tell flang where the netcdf directory is with the "-isystem" flag. But flang ignores it, which causes an error in finding file: netcdf.mod. Is there flang on Hera or Orion?
If so, this would be easier to debug.

module swap gnu7 arm/20.0 source /projects/marvell/setup_env_llvm

rm -rf build ; mkdir build ; cd build export CC=clang export FC=flang export CMAKE_PREFIX_PATH=/opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5

[lmstrin@stria-login2 build]$ cmake -DCMAKE_BUILD_TYPE=release .. -- The Fortran compiler identification is Flang 99.99.1 -- The C compiler identification is Clang 9.0.1 -- Check for working Fortran compiler: /projects/marvell/llvm/9.0.1/bin/flang -- Check for working Fortran compiler: /projects/marvell/llvm/9.0.1/bin/flang -- works -- Detecting Fortran compiler ABI info -- Detecting Fortran compiler ABI info - done -- Checking whether /projects/marvell/llvm/9.0.1/bin/flang supports Fortran 90 -- Checking whether /projects/marvell/llvm/9.0.1/bin/flang supports Fortran 90 -- yes -- Check for working C compiler: /projects/marvell/llvm/9.0.1/bin/clang -- Check for working C compiler: /projects/marvell/llvm/9.0.1/bin/clang -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Found OpenMP_C: -fopenmp=libomp (found version "3.1") -- Could NOT find OpenMP_Fortran (missing: OpenMP_Fortran_FLAGS OpenMP_Fortran_LIB_NAMES) -- Could NOT find OpenMP (missing: OpenMP_Fortran_FOUND Fortran) (found version "3.1") NetCDF_C_INCLUDE_FILE: /opt/atse/libs/arm/openmpi4/netcdf/4.6.3/include/netcdf.h NetCDF_Fortran_INCLUDE_FILE: /opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/include/netcdf.mod NetCDF_C_CONFIG_EXECUTABLE: /opt/atse/libs/arm/openmpi4/netcdf/4.6.3/bin/nc-config NetCDF_Fortran_CONFIG_EXECUTABLE: /opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/bin/nf-config NetCDF_C_LIBRARY: /opt/atse/libs/arm/openmpi4/netcdf/4.6.3/lib/libnetcdf.so NetCDF_Fortran_LIBRARY: /opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/lib/libnetcdff.so -- Found NetCDF: /opt/atse/libs/arm/openmpi4/netcdf/4.6.3/include;/opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/include (found version "4.6.3") found components: C Fortran -- FindNetCDF defines targets: -- - NetCDF_VERSION [4.6.3] -- - NetCDF_PARALLEL [TRUE] -- - NetCDF_C_CONFIG_EXECUTABLE [/opt/atse/libs/arm/openmpi4/netcdf/4.6.3/bin/nc-config] -- - NetCDF::NetCDF_C [SHARED] [Root: /opt/atse/libs/arm/openmpi4/netcdf/4.6.3] Lib: /opt/atse/libs/arm/openmpi4/netcdf/4.6.3/lib/libnetcdf.so -- - NetCDF_Fortran_CONFIG_EXECUTABLE [/opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/bin/nf-config] -- - NetCDF::NetCDF_Fortran [SHARED] [Root: /opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5] Lib: /opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/lib/libnetcdff.so -- Configuring done -- Generating done -- Build files have been written to: /ascldap/users/lmstrin/SENA-c_sw/ref/build [lmstrin@stria-login2 build]$ make VERBOSE=1 /opt/atse/utils/cmake/3.12.2/bin/cmake -H/ascldap/users/lmstrin/SENA-c_sw/ref -B/ascldap/users/lmstrin/SENA-c_sw/ref/build --check-build-system CMakeFiles/Makefile.cmake 0 /opt/atse/utils/cmake/3.12.2/bin/cmake -E cmake_progress_start /ascldap/users/lmstrin/SENA-c_sw/ref/build/CMakeFiles /ascldap/users/lmstrin/SENA-c_sw/ref/build/CMakeFiles/progress.marks make -f CMakeFiles/Makefile2 all make[1]: Entering directory /home/lmstrin/SENA-c_sw/ref/build' make -f src/CMakeFiles/c_sw.dir/build.make src/CMakeFiles/c_sw.dir/depend make[2]: Entering directory/home/lmstrin/SENA-c_sw/ref/build' cd /ascldap/users/lmstrin/SENA-c_sw/ref/build && /opt/atse/utils/cmake/3.12.2/bin/cmake -E cmake_depends "Unix Makefiles" /ascldap/users/lmstrin/SENA-c_sw/ref /ascldap/users/lmstrin/SENA-c_sw/ref/src /ascldap/users/lmstrin/SENA-c_sw/ref/build /ascldap/users/lmstrin/SENA-c_sw/ref/build/src /ascldap/users/lmstrin/SENA-c_sw/ref/build/src/CMakeFiles/c_sw.dir/DependInfo.cmake --color= Dependee "/ascldap/users/lmstrin/SENA-c_sw/ref/build/src/CMakeFiles/c_sw.dir/DependInfo.cmake" is newer than depender "/ascldap/users/lmstrin/SENA-c_sw/ref/build/src/CMakeFiles/c_sw.dir/depend.internal". Dependee "/ascldap/users/lmstrin/SENA-c_sw/ref/build/src/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/ascldap/users/lmstrin/SENA-c_sw/ref/build/src/CMakeFiles/c_sw.dir/depend.internal". Scanning dependencies of target c_sw make[2]: Leaving directory /home/lmstrin/SENA-c_sw/ref/build' make -f src/CMakeFiles/c_sw.dir/build.make src/CMakeFiles/c_sw.dir/build make[2]: Entering directory/home/lmstrin/SENA-c_sw/ref/build' [ 16%] Building Fortran object src/CMakeFiles/c_sw.dir/netCDFModule.f90.o cd /ascldap/users/lmstrin/SENA-c_sw/ref/build/src && /projects/marvell/llvm/9.0.1/bin/flang -DNDEBUG -isystem /opt/atse/libs/arm/openmpi4/netcdf/4.6.3/include -isystem /opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/include -O3 -DNDEBUG -c /ascldap/users/lmstrin/SENA-c_sw/ref/src/netCDFModule.f90 -o CMakeFiles/c_sw.dir/netCDFModule.f90.o clang-9: warning: argument unused during compilation: '-isystem /opt/atse/libs/arm/openmpi4/netcdf/4.6.3/include' [-Wunused-command-line-argument] clang-9: warning: argument unused during compilation: '-isystem /opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/include' [-Wunused-command-line-argument] F90-F-0004-Unable to open MODULE file netcdf.mod (/ascldap/users/lmstrin/SENA-c_sw/ref/src/netCDFModule.f90: 7) F90/aarch64 Linux Flang - 1.5 2017-05-01: compilation aborted make[2]: [src/CMakeFiles/c_sw.dir/netCDFModule.f90.o] Error 1 make[2]: Leaving directory `/home/lmstrin/SENA-c_sw/ref/build' make[1]: [src/CMakeFiles/c_sw.dir/all] Error 2 make[1]: Leaving directory `/home/lmstrin/SENA-c_sw/ref/build' make: *** [all] Error 2 [lmstrin@stria-login2 build]$ ls /opt/atse/libs/arm/openmpi4/netcdf/4.6.3/include netcdf_aux.h netcdf_filter.h netcdf.h netcdf_mem.h netcdf_meta.h netcdf_par.h

christopherwharrop-noaa commented 3 years ago

@LyndStringer - Please edit this issue to include a description of the problem and how to reproduce it.

LyndStringer commented 3 years ago

At the bottom of the long comment, I "ls"ed the wrong directory. Here is the directory that contains netcdf.mod. This was also denoted in an -isystem flag, but ignored, so netcdf.mod was not found.

[lmstrin@stria-login2 build]$ ls /opt/atse/libs/arm/openmpi4/netcdf/4.6.3/include netcdf_aux.h netcdf_filter.h netcdf.h netcdf_mem.h netcdf_meta.h netcdf_par.h

LyndStringer commented 3 years ago

While getting clang/flang working with out kernels is a desirable goal, I doubt we'll be able to get flang/clang working with the kernels by Thursday. There are a number of issues beyond cmake's -isystem problem on stria. So in my mind, this is not urgent.

christopherwharrop-noaa commented 3 years ago

@LyndStringer - You still ls'ed the wrong directory. Does /opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/include contain the netcdf.mod file?

christopherwharrop-noaa commented 3 years ago

@LyndStringer - Also, clang/flang compilers are not currently supported. We don't even have options for that in the build flags

christopherwharrop-noaa commented 3 years ago

This appears to be a bug in flang and/or cmake. See: here.

You could try a newer version of cmake. But, probably, we'll have to find a way to force it to use -I instead of -system.

christopherwharrop-noaa commented 3 years ago

@LyndStringer - Try adding a custom setting for the Fortran flags to add the include path with -I:

cmake -DCMAKE_Fortran_FLAGS=-I/opt/atse/libs/arm/openmpi4/netcdf-fortran/4.4.5/include