search

DrylandEcology / rSFSW2

rSFSW2: A R package to create soil water balance simulation experiment
GNU General Public License v3.0
8 stars 7 forks source link

Reference output database no longer reproduced #377

Closed dschlaep closed 4 years ago

dschlaep commented 4 years ago

Locally and on travis-ci, integration tests (i.e., TestPrj4) were run and compared against reference database dbOutput_TestPrj4_v4.1.0.sqlite3 successfully (i.e., no differences): for instance,

However, they started to disagree with the reference database locally and on travis-ci in Jan 2020. I pulled the commits of rSFSW2, rSOILWAT2, and rSW2utils (if needed) to recreate conditions at several points in time until May 2019. These runs no longer reproduce output as in the reference database. I tested on different machines, under different operating systems, and with different compiler versions.

The table aggregation_overall_mean has 109 differences and aggregation_overall_sd has 97 differences. The daily aggregation tables have 0 differences. I checked intermediate RData files (direct rSOILWAT2 output) and cannot produce any deviations.

Example of differences

$aggregation_overall_mean
  [1] "Component “WaterExtractionSpell_MeanContinuousDuration_L5_days_mean”: Mean absolute difference: 0.2"
  [2] "Component “WaterExtractionSpell_AnnualSummedExtraction_L4_mm_mean”: Mean absolute difference: 0.01"
  [3] "Component “WaterExtractionSpell_AnnualSummedExtraction_L5_mm_mean”: Mean absolute difference: 0.01"
  [4] "Component “DeepDrainage_DailyMax_doy_mean”: Mean absolute difference: 0.01695991"
  [5] "Component “SWP_bottomLayers_DailyMin_doy_mean”: Mean absolute difference: 0.002545719"
  [6] "Component “RelRecharge_bottomLayers_DailyMin_doy_mean”: Mean absolute difference: 0.002555868"
  [7] "Component “NRCS_COND3_MCS_AnyDry_days_mean”: Mean absolute difference: 0.05882353"
  [8] "Component “NRCS_COND7_MCS_AnyMoist_GT180Days_days_mean”: Mean absolute difference: 0.0625"
  [9] "Component “NRCS_COND8_MCS_AnyWetConsec_days_mean”: Mean absolute difference: 0.0625"
 [10] "Component “NRCS_COND10_MCS_AllDry_days_mean”: Mean absolute difference: 0.0625"
 [11] "Component “WetDegreeDays_SWPcrit1500kPa_topLayers_Cdays_mean”: Mean absolute difference: 0.2322581"
 [12] "Component “WetDegreeDays_SWPcrit1500kPa_bottomLayers_Cdays_mean”: Mean absolute difference: 0.3409677"
 [13] "Component “WetDegreeDays_SWPcrit1500kPa_anyLayer_Cdays_mean”: Mean absolute difference: 0.3409677"
 [14] "Component “WetDegreeDays_SWPcrit3000kPa_topLayers_Cdays_mean”: Mean absolute difference: 0.07016129"
 [15] "Component “WetDegreeDays_SWPcrit3000kPa_bottomLayers_Cdays_mean”: Mean absolute difference: 0.1045161"
 [16] "Component “WetDegreeDays_SWPcrit3000kPa_anyLayer_Cdays_mean”: Mean absolute difference: 0.07016129"
 [17] "Component “WetDegreeDays_SWPcrit3900kPa_topLayers_Cdays_mean”: Mean absolute difference: 0.4620968"
 [18] "Component “WetDegreeDays_SWPcrit3900kPa_bottomLayers_Cdays_mean”: Mean absolute difference: 0.03693548"
 [19] "Component “ThermalDrySoilPeriods_SWPcrit1500kPa_NSadj_topLayers_Start_LongestContinuous_days_mean”: Mean absolute difference: 1.032376"