[BUG/ISSUE] RRTMG compiles slowly with CMake with GNU Fortran 9.4.0

[FeiYao-Edinburgh]

What institution are you from?

The University of Edinburgh

Description of the problem

RRTMG compiles slowly with CMake with GNU Fortran 9.4.0. With make -j8 install the compilation takes up to 45 minutes. By contrast, using ifort (IFORT) 2021.5.0 20211109 only takes around 15 minutes. The relevant hints are attached below:

.../rundirs/gc_2x25_47L_merra2_fullchem_RRTMG/CodeDir/src/GEOS-Chem/KPP/fullchem/gckpp_Jacobian.F90:48:0:

   48 | SUBROUTINE Jac_SP ( V, F, RCT, JVS )
      | 
note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without

It pretty resembles one of issues that I bought your attention before but this time on a different subroutine. Do you have any suggestions to speed up the compilation? I will soon test whether the executable from the slow compilation can run smoothly or not.

Description of troubleshooting performed

N/A

GEOS-Chem version

14.1.0

Description of modifications

N/A

Log files

N/A

Software versions

• CMake version: 3.16.3
• Compilers (Intel or GNU, and version): GNU Fortran (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
• NetCDF version: netCDF 4.8.1 & netCDF-Fortran 4.5.2

[lizziel]

Hi @FeiYao-Edinburgh, the slow build time of RRTMG with gfortran is due to files rrtmg_sw_k_g.F90 and rrtmg_lw_k_g.F90. These files contain lots of hard-coded arrays which gfortran is not able to handle as well as ifort, although ifort also goes slow because of them. Since this is 3rd party code and faster compilation can be achieved with ifort, we have not prioritized looking into a fix. I also get the -fvar-tracking-assignments note when building with gfortran but this is not causing the slow-down.

[yantosca]

In the CMakeLists.txt we could try to set -fno-var-tracking-assignments in for the routine gckpp_JacobianSP.F90 file, as this is the one that is being flagged.

[FeiYao-Edinburgh]

I attach the relevant CMakeLists.txt below and would be grateful if you could help set it and get back to me. I am interested in a re-complilation.

CMakeLists.txt

[yantosca]

@FeiYao-Edinburgh @lizziel: See PR #1657 for a fix.

[FeiYao-Edinburgh]

Thanks @yantosca I can confirm that with this fix the warning I described above is gone, although the time required for the compilation is only slightly decreased. The most time-consuming part is:

[ 77%] Built target KPP
[ 77%] Building Fortran object src/GEOS-Chem/GeosRad/CMakeFiles/GeosRad.dir/rrtmg_sw_init.F90.o

But I understand that that's normal for RRTMG.

[yantosca]

@FeiYao-Edinburgh: Yes, I think RRTMG takes a long time to compile, even if you use ifort instead of gfortran. I think it's not so much the compilation but the optimization. It takes less time to compile when we use -DCMAKE_BUILD_TYPE=Debug, since for the debug build it doesn't do any optimization.

[lizziel]

I think the way to get it to build faster is to read the data arrays in via file at run-time rather than hard-code and compile them in rrtmg_sw_k_g.F90 and rrtmg_lw_k_g.F90, one of which is used in rrtmg_sw_init.F90. However, we generally leave it to the community to develop the 3rd party code in the GeosRad folder.