DrylandEcology / STEPWAT2

folder
4 stars 5 forks source link

Inconsistent behavior of generated weather among cells in gridded mode if random seed is 0 #553

Open dschlaep opened 1 year ago

dschlaep commented 1 year ago

I added a script with commit 959f4fcbf0cf060560119ac74b3b62b4198affdc that tests reproducibility expectations as formulated with PR #528 (and see documentation and code comments of set_all_rngs().

I think we should clarify what the expected behavior of weather among gridcells in gridded mode if random seed is 0 should be (a seed of 0 signals non-reproducible random number sequences): 1) Should weather be identical among cells (for a given run) in gridded mode even if random seed is 0?, or 2) Should weather vary among cells in gridded mode if random seed is 0? [The script currently tests for this behavior.]

Either way, the current implementation fails because some gridcells produce identical weather and other gridcells produce (awkwardly [see below]) different weather with a random seed of 0.

I tested this on branch "Seed_Dispersal" (commit b995a9dcd504df62fa2c9f2131f0c156b183be55; June 22, 2023) and on branch "master" (commit 04fb112f5f2d35f6c221010435daebd3853100cf; June 20, 2023).

tools/check_STEPWAT2.sh

-----------------------------------------------------------------------

--- * if seed == 0 (output cannot be reproduced among runs) ------

- Set model.in: 2 100 0 (niter, nyrs, seed)

--- ** weather is different among cells ------

[1] FALSE FALSE TRUE

or

-----------------------------------------------------------------------

--- * if seed == 0 (output cannot be reproduced among runs) ------

- Set model.in: 2 100 0 (niter, nyrs, seed)

--- ** weather is different among cells ------

[1] FALSE TRUE TRUE


*awkwardly different

Rplot Rplot01