OpenMP Detection Assumes GNU-Style Preprocessor Directive for Fortran (Incompatible with IBM XLF)

[nmhamster]

Trilinos OpenMP detection of flags for Fortran compiler does not work correctly with IBM XLF compiler on POWER8 platform. The detection assumes that the -D flag works for passing preprocessor defines through to the compiler. This is not the case for the IBM xlfand xlf90 compilers where -WF,-D needs to be used if we expect the C preprocessor to be called. The correct check should be for -qsmp=omp to be found although its not clear this is correctly tested for (possible I have missed it in the error output).

/home/projects/pwr8-rhel72/ibm/xl/xlf/15.1.3/bin/xlf   -O3 -g -qsmp=omp -qsimd=auto -L/home/projects/pwr8-rhel72/ibm/xl/xlf/15.1.3/lib -L/home/projects/pwr8-rhel72/ibm/xl/xlC/13.1.3/lib -lopen-pal -lxl -lxlopt -lxlf90 -lxlfmath -lm -libmc++ -lstdc++ -L/home/projects/pwr8-rhel72/openmpi/1.10.2/xl/13.1.3/cuda/7.5.7/lib -lmpi  -O3 -g -qsmp=omp -qsimd=auto -L/home/projects/pwr8-rhel72/ibm/xl/xlf/15.1.3/lib -L/home/projects/pwr8-rhel72/ibm/xl/xlC/13.1.3/lib -lxlf90 -DOpenMP_FLAG_DETECTED -mp CMakeFiles/cmTC_48a9b.dir/src.F.o  -o cmTC_48a9b -Wl,-export-dynamic

Yields (incorrect behavior) error of:

ld: warning: cannot find entry symbol nMP_FLAG_DETECTED; defaulting to 0000000010000860

[bartlettroscoe]

CC: @trilinos/framework

Looking at the implementation for this in TriBITS (not sure who wrote this) I see:

  IF(${PROJECT_NAME}_ENABLE_OpenMP)
    FIND_PACKAGE(OpenMP)
    IF(OPENMP_FOUND)
      SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
      SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}")
      # FindOpenMP.cmake doesn't find Fortran flags.  Mike H said this is safe.
      SET(CMAKE_Fortran_FLAGS "${CMAKE_Fortran_FLAGS} ${OpenMP_C_FLAGS}")
    ELSE()
      MESSAGE(FATAL_ERROR "Could not find OpenMP, try setting OpenMP_C_FLAGS and OpenMP_CXX_FLAGS directly")
    ENDIF(OPENMP_FOUND)
  ENDIF(${PROJECT_NAME}_ENABLE_OpenMP)

It seems that updated versions of FindOpenMP.cmake do in fact define an OpenMP_Fortran_FLAGS variable (I looked at CMake 3.5.1).

Si,

Can you please try replacing:

      SET(CMAKE_Fortran_FLAGS "${CMAKE_Fortran_FLAGS} ${OpenMP_C_FLAGS}")

with:

      SET(CMAKE_Fortran_FLAGS "${CMAKE_Fortran_FLAGS} ${OpenMP_Fortran_FLAGS}")

in your copy of Trilinos in the file:

Trilinos/cmake/tribits/core/package_arch/TribitsGlobalMacros.cmake

and see if that works? If it does not work out of the box, then it looks like updated versions of FindOpenMP.cmake allow you to override these as CMake cache vars with -DOpenMP_Fortran_FLAGS="<whatever-you-want>". Just make sure you blow away your CMakeCache and CMakeFiles/ directory to start.

If this works, then we can update TriBITS for this change and snapshot back into Trilinos.

[nmhamster]

@bartlettroscoe I had set the various OpenMP_* flags last night but got caught up by this bug you outlined in my overnight builds. Have made the change and am building now. Recall that the build, link and test on this machine can be greater than 3 days in the current cycle due to very long optimization times. Appreciate your help.

[bartlettroscoe]

Recall that the build, link and test on this machine can be greater than 3 days in the current cycle due to very long optimization times.

Si, since it looks like this error occurs right in the compiler setup phase in the initial CMake configure, you should get feedback pretty fast (before you start building any Trilinos code). Otherwise, after the cmake configure completes (successfully, perhaps after some iteration), then I would build a single Fortran source/object file. For details on how to do that, see:

• https://trilinos.org/docs/files/TrilinosBuildReference.html#building-a-single-object-file
• https://trilinos.org/docs/files/TrilinosBuildReference.html#building-with-verbose-output-without-reconfiguring

[bartlettroscoe]

As described in #1427, setup for OpenMP is controlled by FindOpenMP.cmake which is in the set of standard CMake modules. The solution is to try upgrading CMake and otherwise report the problems to Kitware (and we can make suggested changes).

CC: @trilinos/framework

[github-actions[bot]]

This issue has had no activity for 365 days and is marked for closure. It will be closed after an additional 30 days of inactivity. If you would like to keep this issue open please add a comment and remove the MARKED_FOR_CLOSURE label. If this issue should be kept open even with no activity beyond the time limits you can add the label DO_NOT_AUTOCLOSE.

[github-actions[bot]]

This issue was closed due to inactivity for 395 days.