Open rgknox opened 7 years ago
Just going through my emails, wherein I found this issue. I agree we could test with removing these caps (which were indeed initially put in to avoid a /0 somewhere in the CLM surface exchange code) but maybe after the current crop of run time issues have gone away?
On 14 August 2017 at 19:14, Ryan Knox notifications@github.com wrote:
In EDCanopyStructureMod.F90:calc_areaindex() we impose an artificial minimum on all area indices. In other words, LAI/TAI/etc is not allowed to be lower than 0.1.
I think this is a pretty old artifact in the code, it has been left alone for this long just to retain B4B results in tests, but I think we really should remove this.
Its possible this was originally added to prevent a div0 somewhere, but we can find protections against this.
I could conceive that this minimum is generating some weird inconsistency that is affecting @jkshuman https://github.com/jkshuman 's runs, but this is just a stab in the dark (might be worth testing its removal if no other ideas are generated).
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NGEET/fates/issues/261, or mute the thread https://github.com/notifications/unsubscribe-auth/AMWsQ3AJi3BcNrjM8owN1p7rlQyIfxXmks5sYPDwgaJpZM4O3FAF .
Dr Rosie A. Fisher
Staff Scientist Terrestrial Sciences Section Climate and Global Dynamics National Center for Atmospheric Research 1850 Table Mesa Drive Boulder, Colorado, 80305 USA. +1 303-497-1706 <(303)%20497-1706>
this is the same issue (or a subset anyway) as #101, right? so then that problem still exists?
reopening this because of #739
After checking to see if we could simply remove this again per the discussion https://github.com/NGEET/fates/discussions/1007#discussioncomment-5457762, testing shows that this is still and issue. I'm seeing a floating point error for threaded aux_clm
suite test SMS_D_Lm6_P144x1.f45_f45_mg37.I2000Clm50FatesRs.cheyenne_intel.clm-FatesCold
:
...
49699 102:MPT: #2 MPI_SGI_stacktraceback (
49700 102:MPT: header=header@entry=0x7ffdc2c860d0 "MPT ERROR: Rank 102(g:102) received signal SIGFPE(8).\n\tProcess ID: 55323, Host: r13i3n34, Program: /glade/scratch/glemieux/ctsm-tests /tests_ai-min-remove-aux-3/SMS_D_Lm6_P144x1.f45_f45_mg37.I2000Clm50"...) at sig.c:340
49701 102:MPT: #3 0x00002b2d04b8fe66 in first_arriver_handler (signo=signo@entry=8,
49702 102:MPT: stack_trace_sem=stack_trace_sem@entry=0x2b2d14480080) at sig.c:489
49703 102:MPT: #4 0x00002b2d04b900f3 in slave_sig_handler (signo=8, siginfo=<optimized out>,
49704 102:MPT: extra=<optimized out>) at sig.c:565
49705 102:MPT: #5 <signal handler called>
49706 102:MPT: #6 0x00000000018ed580 in edsurfaceradiationmod::ed_sunshadefracs (nsites=10,
49707 102:MPT: sites=..., bc_in=..., bc_out=...)
49708 102:MPT: at /glade/u/home/glemieux/ctsm/src/fates/biogeophys/EDSurfaceAlbedoMod.F90:1266
...
The fates
suite test ERS_Lm13.f10_f10_mg37.I2000Clm50Fates.cheyenne_gnu.clm-FatesCold
is failing during the TotalBalanceCheck
call due to aresp_acc
being NaN
:
5751 49: ENDRUN:ERROR in /glade/u/home/glemieux/ctsm/src/fates/main/EDMainMod.F90 at line 1013
...
65286 49: mass balance error detected
65287 49: element type (see PRTGenericMod.F90): 1
65288 49: error fraction relative to biomass stock: 0.0000000000000000
65289 49: absolut error (flux in - change): NaN
65290 49: call index: 1
65291 49: Element index (PARTEH global): 1
65292 49: net: NaN
65293 49: dstock: -0.16263594704082607
65294 49: seed_in: 0.0000000000000000
65295 49: net_root_uptake: 0.0000000000000000
65296 49: gpp_acc: 1.8726980289346781E-003
65297 49: flux_generic_in: 0.0000000000000000
65298 49: wood_product: 0.0000000000000000
65299 49: error from patch resizing: 0.0000000000000000
65300 49: burn_flux_to_atm: 0.0000000000000000
65301 49: seed_out: 0.0000000000000000
65302 49: flux_generic_out: 0.0000000000000000
65303 49: frag_out: 1.4318040557578071E-002
65304 49: aresp_acc: NaN
65305 49: error=net_flux-dstock: NaN
65306 49: biomass 293.76402263977747
65307 49: litter 663.48754971364178
65308 49: seeds 9.8703240123981963
65309 49: total stock 967.12189636581752
65310 49: previous total 967.28453231285835
65311 49: lat lon -70.000000000000000 15.000000000000000
65312 49: ---------------------------------------
@glemieux , for the SIGFPE(8) fail, can you confirm that line 1266 in the error matches this line from the recent tag:
https://github.com/NGEET/fates/blob/sci.1.65.3_api.25.4.0/biogeophys/EDSurfaceAlbedoMod.F90#L1266
@rgknox confirmed :+1:
In EDCanopyStructureMod.F90:calc_areaindex() we impose an artificial minimum on all area indices. In other words, LAI/TAI/etc is not allowed to be lower than 0.1.
I think this is a pretty old artifact in the code, it has been left alone for this long just to retain B4B results in tests, but I think we really should remove this.
Its possible this was originally added to prevent a div0 somewhere, but we can find protections against this.
I could conceive that this minimum is generating some weird inconsistency that is affecting @jkshuman 's runs, but this is just a stab in the dark (might be worth testing its removal if no other ideas are generated).