[BUG/ISSUE] Invalid compile command for hemco_standalone.x on Ubuntu

[JiaweiZhuang]

At the very end of compilation, hemco_standalone.x is compiled by:

mpifort -cpp -w -std=legacy -fautomatic -fno-align-commons -fconvert=native -fno-range-check -O3 -funroll-loops -mcmodel=medium -fbacktrace -DLINUX_GFORTRAN -DEXTERNAL_GRID -DNC_DIAG -DGEOS_FP -DNC_HASCOMPRESSION -DESMF -DUSE_REAL8 hcoi_esmf_mod.o hemco_standalone.o hcoi_standalone_mod.o -L../../lib -lHCOI -lHCOX -lHCO -lGeosUtil -lHeaders -lNcUtils -L/usr/lib -lnetcdff -L/usr/lib -lnetcdf -lnetcdf -L/usr/lib -lnetcdf -L/home/ec2-user/tutorial/gchp_standard/CodeDir/GCHP/Shared/Linux/lib -lMAPL_Base -lMAPL_cfio -lGMAO_mpeu -lGMAO_pilgrim -L/home/ec2-user/tutorial/gchp_standard/CodeDir/GCHP/Shared/Linux/lib -lFVdycoreCubed_GridComp -lfvdycore -lGFDL_fms -lGEOS_Shared -lGMAO_hermes -lrt /home/ec2-user/tutorial/gchp_standard/CodeDir/GCHP/ESMF/Linux/lib/libesmf.so -L/usr/local/mpich/bin/../lib64 -lmpich -lmpichf90 -o hemco_standalone.x

This runs on CentOS but crashes on Ubuntu (the extremely common undefined reference to `__netcdf_MOD_nf90_create' error), because Ubuntu requires -lnetcdff -lnetcdf to appear after the object that uses it (https://stackoverflow.com/a/13953322/8729698).

This can be fixed by adding -lnetcdf -lnetcdff to the very end of the command:

mpifort -I/usr/include -cpp -w -std=legacy -fautomatic -fno-align-commons -fconvert=native -fno-range-check -O3 -funroll-loops -mcmodel=medium -fbacktrace -DLINUX_GFORTRAN -DEXTERNAL_GRID -DNC_DIAG -DBPCH_TPBC -DGEOS_FP -DNC_HASCOMPRESSION -DESMF -DUSE_REAL8 hcoi_esmf_mod.o hemco_standalone.o hcoi_standalone_mod.o -L../../lib -lHCOI -lHCOX -lHCO -lGeosUtil -lHeaders -lNcUtils -L/usr/lib -lnetcdff -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -lnetcdf -lnetcdf -L/usr/lib/x86_64-linux-gnu -L/usr/lib/x86_64-linux-gnu/hdf5/serial -lhdf5_hl -lhdf5 -lpthread -lsz -lz -ldl -lm -lcurl -L/home/ubuntu/tutorial/gchp_standard/CodeDir/GCHP/Shared/Linux/lib -lMAPL_Base -lMAPL_cfio -lGMAO_mpeu -lGMAO_pilgrim -L/home/ubuntu/tutorial/gchp_standard/CodeDir/GCHP/Shared/Linux/lib -lFVdycoreCubed_GridComp -lfvdycore -lGFDL_fms -lGEOS_Shared -lGMAO_hermes -lrt /home/ubuntu/tutorial/gchp_standard/CodeDir/GCHP/ESMF/Linux/lib/libesmf.so -L/usr/bin/../lib64 -lmpich -lmpichf90 -lnetcdf -lnetcdff -o hemco_standalone.x

But how to add this to the Makefile gets a bit tricky.

A simpler way is to skip the compilation of hemco_standalone.x, since it is not used in GCHP anyway (does it even work?).

This can be done by changing the dependency in HEMCO/Interfaces/Makefile from

all:
        @${MAKE} lib
        @${MAKE} exe

to

all:
        @${MAKE} lib

• Here's the successful compile log with hemco_standalone.x skipped: compile_no_hemco_success.log
• Here's the original, failed compile log: compile_hemco_error.log

My question is

1. Can we skip hemco_standalone.x by default?
2. How to do this only for GCHP while not affect GC-classic?

[JiaweiZhuang]

A better way to skip hemco_standalone.x is changing HEMCO/Makefile:

from

all:
    @$(MAKE) lib

...

lib:
    @$(MAKE) libHCO
    @$(MAKE) libHCOX
    @$(MAKE) libHCOI
    @$(MAKE) exe

to

all:
    @$(MAKE) lib
    @$(MAKE) exe
...

lib:
    @$(MAKE) libHCO
    @$(MAKE) libHCOX
    @$(MAKE) libHCOI

This feels a lot more intuitive as lib only contains object files while all also contains executable.

Right now the above all target is not used at all. The top-level GeosCore/Makefile only goes to lib:

libhemco:                                          # Build code in HEMCO/*
    @$(MAKE) -C $(HEMCO) lib

So, hemco_standalone.x will not be compiled in any case, for both classic and HP.

If users do want to get hemco_standalone.x, the above unused all can be used to implement one more option, like:

hemco_standalone:                                          # Build hemco_standalone.x
    @$(MAKE) -C $(HEMCO) all

[lizziel]

Thanks for coming up with this solution. In the past I adjusted the makefile during debugging to skip compiling the HEMCO standalone since it is indeed not used. It makes sense to make this the default. I will keep this issue open until we put it into an upcoming version and test it.

[yantosca]

I pushed a fix for this. We use the Makefile_header.mk variable IS_HPC to prevent building the hemco_standalone.x executable if we are building GC for an HPC environment.

    @$(MAKE) libHCO
    @$(MAKE) libHCOX
    @$(MAKE) libHCOI
ifeq ($(IS_HPC),0)
    @$(MAKE) exe
endif