search

NGEET / fates

repository for the Functionally Assembled Terrestrial Ecosystem Simulator (FATES)
Other
105 stars 92 forks source link

anatomy of a cesm.log #858

Open rgknox opened 2 years ago

rgknox commented 2 years ago

I ran a 60 year f45 simulation with FATES-CLM with no reduced compexity. I then took the cesm log from the last 10 years and evaluated the frequency of error messages reported. The model configuration does not use the medlyn conductance, so it does not harbour the same issues with the medlyn solver error we are encountering, as reported here: #852.

message portion calling location notes counts
BalanceCheck, solar radiation balance error link writes 3 lines of text 14,430,847
dbh counter link counter ~ 12,13 1,035,323
sysmem size=67286.6 MB rss=288.2 ... ./cime/src/share/timing/GPTLprint_memusage.c 3651
Negative conc. in ch4tran link 513
Note: sink > source in ch4_tran link 27
ED: issue with currentPatch link 19
36:GPTLprint_memusage: Using Kbytesperpage=4 during run-time, not init 1
adrifoster commented 1 year ago

@rgknox was there plans to try to reduce the frequency of these errors? I can't remember what happened to those ideas. I'm currently dealing with this issue again for a global f45 full FATES run. Things are incredibly slow!

adrifoster commented 1 year ago

My issue right now is the solar radiation balance check. Perhaps I can just turn it off? Or up the tolerance for now?

rgknox commented 1 year ago

This PR addressed some of this stuff: #792

That DBH counter warning now will only report so many times since its now wrapped in fateswarn(): https://github.com/NGEET/fates/blob/main/biogeochem/FatesAllometryMod.F90#L2528

But we have not addressed this radiation balance error, and as far as I know there is no agreed upon solution.