Output depends on the time when the code is executed

[dschlaep]

STEPWAT2 produces (slightly) different outputs depending on the time the code is executed -- independent of inputs. The output varies depending on whether year (when the simulation is run) is a leap year or not.

This issue is caused by a bug in SOILWAT2, see https://github.com/DrylandEcology/SOILWAT2/issues/273

To reproduce this error: (i) change your time or (ii) modify function sw_src/Times.c/Time_now(). Here code for the latter option:

# Produce "master-branch" output (dependent on user's current year):
make cleanbin bint_testing_nongridded
cp -r testing.sagebrush.master/Stepwat_Inputs/Output testing.sagebrush.master/Stepwat_Inputs/Output_ref

# Modify SOILWAT2 function `Time_now()` to return a fixed time of a non-leap year (here, 1970):
void Time_now(void) {
    /* =================================================== */
/*
    time_t x = time(NULL );

    memcpy(&_tym, (struct tm *) localtime(&x), sizeof(struct tm));
*/
  // Initialize to Jan 1, 1970
  _tym.tm_year = 70;
  _tym.tm_mon = 0;
  _tym.tm_mday = 1;
  _tym.tm_hour = 0;
  _tym.tm_min = 0;
  _tym.tm_sec = 0;
  mktime(&_tym);

    _reinit();
}

# Compile and produce output
make cleanbin bint_testing_nongridded
cp -r testing.sagebrush.master/Stepwat_Inputs/Output testing.sagebrush.master/Stepwat_Inputs/Output_ref1970

# Modify function `Time_now()` to return a fixed time of a leap year (here, 1980):
...
  _tym.tm_year = 80;
...

# Compile and produce output
make cleanbin bint_testing_nongridded
cp -r testing.sagebrush.master/Stepwat_Inputs/Output testing.sagebrush.master/Stepwat_Inputs/Output_ref1980

# Modify function `Time_now()` to return a fixed time of another non-leap year (here, 1990):
...
  _tym.tm_year = 90;
...

# Compile and produce output
make cleanbin bint_testing_nongridded
cp -r testing.sagebrush.master/Stepwat_Inputs/Output testing.sagebrush.master/Stepwat_Inputs/Output_ref1990

# Compare output
cd testing.sagebrush.master/Stepwat_Inputs/
diff Output_ref/ Output_ref1970/ -rqs
diff Output_ref/ Output_ref1980/ -rqs
diff Output_ref1970/ Output_ref1980/ -rqs
diff Output_ref1970/ Output_ref1990/ -rqs
cd ../..

--> depending on user time leap/non-leap year, Output_ref will be either identical to Output_ref1970 (non-leap year) or to Output_ref1980 (leap year), but not both because Output_ref1970 and Output_ref1980 are different (despite identical seed) -- whereas Output_ref1970 and Output_ref1990 are identical (both non-leap years)

diff Output_ref1970/ Output_ref1980/ -rqs
Files Output_ref1970/bmassavg.csv and Output_ref1980/bmassavg.csv differ
Files Output_ref1970/mortavg.csv and Output_ref1980/mortavg.csv are identical
Files Output_ref1970/sw_output/sw2_daily_agg.csv and Output_ref1980/sw_output/sw2_daily_agg.csv differ
Files Output_ref1970/sw_output/sw2_daily_rep1.csv and Output_ref1980/sw_output/sw2_daily_rep1.csv differ
Files Output_ref1970/sw_output/sw2_daily_slyrs_agg.csv and Output_ref1980/sw_output/sw2_daily_slyrs_agg.csv differ
Files Output_ref1970/sw_output/sw2_daily_slyrs_rep1.csv and Output_ref1980/sw_output/sw2_daily_slyrs_rep1.csv differ
Files Output_ref1970/sw_output/sw2_monthly_agg.csv and Output_ref1980/sw_output/sw2_monthly_agg.csv differ
Files Output_ref1970/sw_output/sw2_monthly_rep1.csv and Output_ref1980/sw_output/sw2_monthly_rep1.csv differ
Files Output_ref1970/sw_output/sw2_monthly_slyrs_agg.csv and Output_ref1980/sw_output/sw2_monthly_slyrs_agg.csv differ
Files Output_ref1970/sw_output/sw2_monthly_slyrs_rep1.csv and Output_ref1980/sw_output/sw2_monthly_slyrs_rep1.csv differ
Files Output_ref1970/sw_output/sw2_yearly_agg.csv and Output_ref1980/sw_output/sw2_yearly_agg.csv differ
Files Output_ref1970/sw_output/sw2_yearly_rep1.csv and Output_ref1980/sw_output/sw2_yearly_rep1.csv differ
Files Output_ref1970/sw_output/sw2_yearly_slyrs_agg.csv and Output_ref1980/sw_output/sw2_yearly_slyrs_agg.csv differ
Files Output_ref1970/sw_output/sw2_yearly_slyrs_rep1.csv and Output_ref1980/sw_output/sw2_yearly_slyrs_rep1.csv differ

diff Output_ref1970/ Output_ref1990/ -rqs
Files Output_ref1970/bmassavg.csv and Output_ref1990/bmassavg.csv are identical
Files Output_ref1970/mortavg.csv and Output_ref1990/mortavg.csv are identical
Files Output_ref1970/sw_output/sw2_daily_agg.csv and Output_ref1990/sw_output/sw2_daily_agg.csv are identical
Files Output_ref1970/sw_output/sw2_daily_rep1.csv and Output_ref1990/sw_output/sw2_daily_rep1.csv are identical
Files Output_ref1970/sw_output/sw2_daily_slyrs_agg.csv and Output_ref1990/sw_output/sw2_daily_slyrs_agg.csv are identical
Files Output_ref1970/sw_output/sw2_daily_slyrs_rep1.csv and Output_ref1990/sw_output/sw2_daily_slyrs_rep1.csv are identical
Files Output_ref1970/sw_output/sw2_monthly_agg.csv and Output_ref1990/sw_output/sw2_monthly_agg.csv are identical
Files Output_ref1970/sw_output/sw2_monthly_rep1.csv and Output_ref1990/sw_output/sw2_monthly_rep1.csv are identical
Files Output_ref1970/sw_output/sw2_monthly_slyrs_agg.csv and Output_ref1990/sw_output/sw2_monthly_slyrs_agg.csv are identical
Files Output_ref1970/sw_output/sw2_monthly_slyrs_rep1.csv and Output_ref1990/sw_output/sw2_monthly_slyrs_rep1.csv are identical
Files Output_ref1970/sw_output/sw2_yearly_agg.csv and Output_ref1990/sw_output/sw2_yearly_agg.csv are identical
Files Output_ref1970/sw_output/sw2_yearly_rep1.csv and Output_ref1990/sw_output/sw2_yearly_rep1.csv are identical
Files Output_ref1970/sw_output/sw2_yearly_slyrs_agg.csv and Output_ref1990/sw_output/sw2_yearly_slyrs_agg.csv are identical
Files Output_ref1970/sw_output/sw2_yearly_slyrs_rep1.csv and Output_ref1990/sw_output/sw2_yearly_slyrs_rep1.csv are identical

[kpalmqui]

@dschlaep I am very sorry to hear this! What a pain. On a positive note, this bug appears to have not much affect on biomass within STEPWAT2.

Year = 1970 1 iter seed 7, mean total biomass = 544.059

Year = 1980 1 iter seed 7, mean total biomass = 544.157

I also examined output for 100 iters (but of course this is less directly comparable because of potentially different random seeds between runs)

Year = 1970 100 iter seed 0, mean total biomass = 573.24

Year = 1980 100 iter seed 0, mean total biomass = 581.30