[BUG/ISSUE] KPP solver error when trying to reduce tolerances for SOA simulations

[yantosca]

I am adding this issue on behalf of Katie Travis (katherine.travis@nasa.gov):

I am trying to use this suggestion on the wiki to debug an error with GC.12.6.0 for a nested Asia simulation with MERRA2 (0.5x0.625). It runs fine at 2x25.

--> Step size too small: T + 10*H = T or H < Roundoff
T=   904.261455276099      and H=  5.153668929013635E-013
2.220446049250313E-016
## INTEGRATE FAILED TWICE !!! 
===============================================================================
GEOS-CHEM ERROR: Integrator error code : -7
STOP at INTEGRATE_KPP
===============================================================================

I have kept tightening RTOL down to 5E-4 with no luck. I get the following out in kpp_debug_007.txt

TIME                 NaN
SUN       0.00000000E+00
TEMP      2.81566232E+02
RTOLS                NaN
TSTART    0.00000000E+00
TEND      1.20000000E+03
DT        1.20000000E+03
STEPMIN   1.58178035E+02
STEPMAX              NaN
CFACTOR   1.00000000E+00

I don’t understand the error output, would you mind helping me interpret it?

[yantosca]

Thanks for writing. The output that you list above is generated deep into KPP,in the KPP/*/gckpp_Integrator.F90 module.

   IF ( ((T+0.1_dp*H) == T).OR.(H <= Roundoff) ) THEN  ! Step size too small
      CALL ros_ErrorMsg(-7,T,H,IERR)
      RETURN
   END IF

It seems to indicate that the internal step size got too small for KPP to solve at a particular grid box.

I suspect there is some issue with concentrations being very high within the grid box. Maybe this can happen at the higher resolution of the nested grid.

Are you using e.g. the online dust and aerosols or the offline versions? At very high resolutions the online dust can cause emissions that are very high. That might be something to look into.

[yantosca]

I am going to close out this issue for now. Feel free to reopen it.

[theloniuspunk]

Hi folks,

Is there a fix to this issue? I just had it as well. 0.5x0.667 Nested NA sim with MERRA-2. Full Chem (not TOMAS). It happened 12 months into the simulation.

Jeff

[yantosca]

Hi @theloniuspunk. We are not aware of a fix for this as of yet. I've reopened the issue.

[theloniuspunk]

Thanks, Bob!

On Fri, Feb 12, 2021 at 3:17 PM Bob Yantosca notifications@github.com wrote:

Hi @theloniuspunk https://github.com/theloniuspunk. We are not aware of a fix for this as of yet. I've reopened the issue.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/geoschem/geos-chem/issues/191#issuecomment-778484786, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQBKQDUIKJKLO5QVIRSYOR3S6WSGVANCNFSM4KD6HCNQ .

[luhu0]

I had the same issue running 201807 with the 0.25x0.3125 nested NA + GEOS-FP + fullchem in v13 as well. The simulation stopped after ~15 mins, when GC seems to still read emission inputs. I tried to turn off all offline emissions, and change dates, the issue remained. Thanks for any inputs and suggestions. Lu

[yantosca]

Thanks for writing @luhu0. Can you try configuring GEOS-Chem with

cmake -DCMAKE_BUILD_TYPE=Debug

that will turn on the debugging flags. If the problem is showing up in the 1st 15 minutes of a run, you could set up the input.geos. etc. files for a short run (like an hour) and then see if any errors turn up.

[luhu0]

Thanks, Bob. I tried the Debug option (with GCC 9.2.0). It looks like the error shows up right after Begin Time Stepping. I've also tested the versions in both fullchem and tropchem in the v13 nested NA, they all showed the same issue. See attached for the log file. GC.log

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If there are no updates within 7 days it will be closed. You can add the "never stale" tag to prevent the Stale bot from closing this issue.

[stale[bot]]

Closing due to inactivity

[LukeFakes]

I'm having a similar issue running with a custom nested grid in v13.1.2, if anyone has had an luck with it, let me know please!

[mzhang78]

I'm having a similar issue running with a nested NA region v12.9.3 tropchem. (20160108) Thanks

[yantosca]

@mzhang78, we now recommend not using the tropchem simulation if possible, but using fullchem with 47 levels instead. See #839.

[arianatribby]

@yantosca I am having a similar problem. I am attempting a nested run with a full chemistry run, custom grid, 0.5x0.625 resolution on AWS, gcclassic v13.4.1.

GEOS-Chem ERROR: KPP failed to converge after 2 iterations!
 -> at Do_FullChem (in module GeosCore/FullChem_mod.F90)

I also noticed additional warnings before the error:

My edit 2, assigning -val to 0
 WARNING: Negative concentration for species H2O at (I,J,L) =          120           9          30
My edit 2, assigning -val to 0
 WARNING: Negative concentration for species O3 at (I,J,L) =          117          97    

This makes me wonder if it is the boundary conditions file, but I generated it as instructed. I also tried turning on online dust like suggested but it didn't change the result.

input.geosnested.txt HEMCO_Config.rcnested.txt run.log: https://caltech.box.com/s/7w0lys8n537t33p3fd2wq3sdb7zl6k83 boundary conditions: https://caltech.box.com/s/rkr7f5caxsj3sxbi1s2532esb4o1x8zn

Thanks for your help.

[yantosca]

@arianatribby, would you mind opening a new issue for this? Thanks.