Remove nan flag for nag.

[fischer-ncar]

Several tests were failing because of the -nan flag being used with the nag compiler. So backing that change out.

[jedwards4b]

The cam folks were pretty insistent about adding this flag, perhaps they should review?

[fischer-ncar]

I'll add @cacraigucar.

[cacraigucar]

Is there perhaps a way we can have a special flag that turns on the -nan for those of us who want to test with it? Right now I'm editing the file and that works ok, but it would be nice not to have to do that

[cacraigucar]

Expanding on my previous comment, we need some way to be able to turn on the -nan flag so that we can set up some regression tests to use it. Hand editing won't be an option for regression testing

[jedwards4b]

I can't think of an easy way to do this right now. I don't think that you can use a test mod because those are done before case.setup. We could come up with a more elaborate method in which we add a ADDFCFLAGS variable to the build flags.

[cacraigucar]

I have verified that a CAMDEN (CAM-SIMA) run using Kessler runs to completion with the -nan flag, hence our request. As we are using ccs_config along with the entire CESM setup structure for CAMDEN, we need this ability somehow in a way that can be automated. We want to make sure as we build up the CAMDEN model, that we maintain that everything is initialized as we add functionality. Since we will be starting that process very soon, we will want to have this feature.

[jedwards4b]

Here is a way you can do that: Edit file Depends.nag and add code like:

ADD_NAN_OBJS=\
somefile.o \
anotherfile.o \
etc.o

ifeq ($(DEBUG),TRUE)
  $(ADD_NAN_OBJS):  %.o: %.F90
      $(FC) -c $(INCLDIR) $(INCS) $(FFLAGS) $(FREEFLAGS) -nan
endif

[cacraigucar]

Can a custom Depends.nag file be brought in via a testmods directory somehow? Perhaps a cp in shell commands?

[jedwards4b]

Sure you can do that, just drop it in the case directory.

[fischer-ncar]

Copying a custom Depends.nag as a testmod looks like an issue for another PR. But for now I would like to go ahead and remove the -nan.

[billsacks]

I like the idea of using this flag if it isn't too hard to get things passing with it. To that end, I have opened a CTSM issue for the error that I observed when trying it: https://github.com/ESCOMP/CTSM/issues/1994

What are people's sense of the priority of fixing this in CTSM and elsewhere? @cacraigucar do you know if CTSM was the main component responsible for problems in your testing? (I'm wondering how far we'd get by trying to solve this on the CTSM side... or is that just a small piece of the problem with enabling this flag?)

[cacraigucar]

@billsacks - I've not gotten as far as CTSM because the CAM job I was concentrating on was always halting in ESMF. I suspect that CAM is passing a bad value to ESMF, but I tried initializing the suspect variable everywhere I saw it being used and none of those places fixed the problem. I did verify that our limited code in the next version of CAM (CAMDEN aka CAM-SIMA) currently runs to completion with -nan set. For now we are going make sure as we port physics over to the next version that we do not break running with -nan. That is CAM's backup approach for now. So it will be a priority moving forward, but we are abandoning it for now as we just don't have time to dedicate to fixing it right now.

[billsacks]

Thanks for the info @cacraigucar . In that case I'm taking it off of CTSM's high priority list, but will keep this open to try to resolve at some point.