Get mksurfdata_esmf tool working on Casper and Izumi

[slevis-lmwg]

Brief summary of bug

The tool does not build on casper and izumi. Continuing from #2201 which included derecho in the discussion. The tool now works on Derecho, so the issue drops in priority for now.

General bug information

CTSM version you are using: I have not tried with the current version (alpha-ctsm5.2.mksrf.19_ctsm5.1.dev163), but I expect the same or similar behavior on casper and izumi as was reported previously in #2201.

Does this bug cause significantly incorrect results in the model's science? The tool does not build on casper and izumi.

[slevis-lmwg]

Since derecho is down, I experimented some on izumi and eventually succeeded in creating an executable on izumi. I may still run into trouble because I got this warning:

ld: warning: libnetcdf.so.19, needed by /project/esmf/PROGS/mvapich2/2.3.3/intel/20.0.1/parallelio/2.5.10/lib/libpiof.so, may conflict with libnetcdf.so.18
[100%] Built target mksurfdata

Successfully created mksurfdata_esmf executable for: izumi_intel for mvapich2 library

I will report when I know.

Either way, here's what I had to do in CMakeLists.txt:

-set_property(TARGET pioc PROPERTY IMPORTED_LOCATION $ENV{PIO}/lib/libpioc.so)
-set_property(TARGET piof PROPERTY IMPORTED_LOCATION $ENV{PIO}/lib/libpiof.so)
+set_property(TARGET pioc PROPERTY IMPORTED_LOCATION /project/esmf/PROGS/mvapich2/2.3.3/intel/20.0.1/parallelio/2.5.10/lib/libpioc.so)
+set_property(TARGET piof PROPERTY IMPORTED_LOCATION /project/esmf/PROGS/mvapich2/2.3.3/intel/20.0.1/parallelio/2.5.10/lib/libpiof.so)

[slevis-lmwg]

Next sequence of commands:

module unload lang/python/3.7.0`
./gen_mksurfdata_namelist --res 10x15 --start-year 1850 --end-year 1850 --rawdata-dir /fs/cgd/csm/inputdata`
./download_input_data
./gen_mksurfdata_jobscript_single --number-of-nodes 1 --tasks-per-node 48 --namelist-file surfdata_10x15_hist_1850_78pfts_c240410.namelist

The last one failed with ERROR: Could not find a matching MPI for attributes: {'mpilib': 'default'} so I used a preexisiting mksurfdata_jobscript that I had from a while ago BUT the run failed with ./tool_bld/mksurfdata: error while loading shared libraries: libpiof.so: cannot open shared object file: No such file or directory

I tried changing tool_bld/.env_mach_specific.sh from mvapich2/2.3.3/intel/20.0.1/pio/2_5_10 to mvapich2/2.3.3/intel/20.0.1/parallelio/2_5_10 to match my change in CMakeLists.txt, but now the run failed with

mvapich2/2.3.3/intel/20.0.1/parallelio(50):ERROR:105: Unable to locate a modulefile for 'mvapich2/2.3.3/intel/20.0.1/parallelio/2_5_10'
./tool_bld/mksurfdata: error while loading shared libraries: libpiof.so: cannot open shared object file: No such file or directory

The modulefile seems to me to exist but permissions are more restrictive for this one versus the one that we used previously: -rw-rw-r-- 1 jedwards cgdcsm 1610 Jan 24 2023 2.5.10 -rwxr-xr-x 1 fischer cgdcsm 969 Mar 1 2023 2_5_10 @jedwards4b you seem to own the one that I'm trying to load. Could you make this modulefile accessible for loading, and I hope that my problems will be solved :-) mvapich2/2.3.3/intel/20.0.1/parallelio/2_5_10

[jedwards4b]

It looks like Chris Fischer owns that one - also I strongly recommend updating to pio2.6.2 which is the latest tag.

[ekluzek]

@jedwards4b thanks for that comment about the update. @slevis-lmwg you should try this in one of the more recent ctsm5.2 tags using ctsm5.1.dev172 or after. That will bring the externals update that @jedwards4b mentions.

[ekluzek]

This won't happen by ctsm5.3 but maybe by the release?