Add pFUnit support on derecho with the intel compiler

[billsacks]

I built and installed pFUnit as follows:

git clone git@github.com:Goddard-Fortran-Ecosystem/pFUnit.git
cd pFUnit
git describe --tags
# that shows v4.8.0
export PFUNIT=$CESMDATAROOT/tools/pFUnit/pFUnit4.8.0_derecho_Intel2023.2.1_noMPI_noOpenMP
mkdir build-dir
cd build-dir
/glade/work/sacks/ctsm_code/ctsm/cime/CIME/scripts/configure --mpilib mpi-serial
source ./.env_mach_specific.sh
cmake -DSKIP_MPI=YES -DSKIP_OPENMP=YES -DCMAKE_INSTALL_PREFIX=$PFUNIT ..
make -j 8
make tests
make install

Note that this used the following modules:

Currently Loaded Modules:
  1) cesmdev/1.0   (H,S)   3) craype/2.7.23    5) mkl/2023.2.0          7) cmake/3.26.3       9) hdf5/1.12.2   11) parallelio/2.6.2
  2) ncarenv/23.09 (S)     4) intel/2023.2.1   6) ncarcompilers/1.0.0   8) mpi-serial/2.3.0  10) netcdf/4.9.2  12) esmf/8.6.0b04

I tested in the context of the latest CTSM master by running the following from the src directory with this change in place:

../cime/scripts/fortran_unit_testing/run_tests.py --build-dir unit_tests.temp

All CTSM unit tests built & passed.

Resolves #131 - fyi @ekluzek

[jedwards4b]

When run on cime I am getting an error:

CMake Error at /glade/work/jedwards/sandboxes/cesm2_x_alpha/share/test/unit/dynamic_vector/CMakeLists.txt:44 (make_cpp_command):
  Unknown CMake command "make_cpp_command".

any ideas?

[jedwards4b]

I've spent the afternoon on it but still can't get the scripts_regression_tests.py test_sys_unittest.py to pass. I hope @billsacks can take a stab at it tomorrow.

[jedwards4b]

You need these changes in ccs_config to use pfunit diff --git a/machines/cmake_macros/intel_derecho.cmake b/machines/cmake_macros/intel_derecho.cmake index 7efd2ee..cf2a10e 100644 --- a/machines/cmake_macros/intel_derecho.cmake +++ b/machines/cmake_macros/intel_derecho.cmake @@ -1,4 +1,5 @@ set(CONFIG_ARGS "--host=cray") +set(PFUNIT_PATH "$ENV{CESMDATAROOT}/tools/pFUnit/pFUnit4.8.0_derecho_Intel2023.2.1_noMPI_noOpenMP") string(APPEND CFLAGS " -qopt-report -march=core-avx2") string(APPEND FFLAGS " -qopt-report -march=core-avx2") if (COMP_NAME STREQUAL gptl)

diff --git a/machines/config_machines.xml b/machines/config_machines.xml
index 7173046..5dd6109 100644
--- a/machines/config_machines.xml
+++ b/machines/config_machines.xml
@@ -1352,12 +1352,12 @@ This allows using a different mpirun command to launch unit tests

       <modules DEBUG="TRUE">
        <command name="load">parallelio/2.6.2-debug</command>
-       <command name="load">esmf/8.6.0b04-debug</command>
+       <command name="load">esmf/8.6.0-debug</command>
       </modules>

       <modules DEBUG="FALSE">
        <command name="load">parallelio/2.6.2</command>
-       <command name="load">esmf/8.6.0b04</command>
+       <command name="load">esmf/8.6.0</command>
       </modules>
     </module_system>

You also need the cime branch fix_test_sys_unittest.py from my fork.

[billsacks]

I hope @billsacks can take a stab at it tomorrow.

Well, I've gotten it working with a bit more destruction than simply stabbing - more like the nuclear option on a portion of the unit tests 😀. I was getting failures in the cpl7 unit test build and in building the mct library for the cpl7 unit tests. It feels like those aren't important anymore, so I simply stopped building that stuff and now the scripts_regression_tests unit tests pass for me. I'll open associated CIME and share PRs shortly with the necessary changes.

[billsacks]

@jedwards4b - the changes I needed are here:

• https://github.com/ESMCI/cime/pull/4536
• https://github.com/ESCOMP/CESM_share/pull/41

I did not need any of the other changes you mentioned in your recent comment, but maybe those are needed for other purposes?

Note that I ran these in the context of latest CTSM master – i.e., cloned ctsm master, ran manage_externals, updated my cime, ccs_config and share externals to these three branches, then ran ./scripts_regression_tests.py test_sys_unittest.py from cime/CIME/tests/. @jedwards4b do you want to test this combination of changes and see if they work for you, too?

[billsacks]

(@ekluzek you should not need those cime and share updates if all you care about is getting the CTSM unit tests to pass: those are just needed for the CIME / share unit testing.)