conform random seed replicability on all systems/versions

[sdtaylor]

The random spatial stuff in flowering_gradient_generator.R (specifically generate_random_spatial_gradient() is fairly finicky and the analysis won't work unless random things can be reproduced with a set seed.

R 3.6 uses the seed differently and thus results across versions are not reproducible.

see Stackoverflow: Is set.seed consistent over different versions of R (and Ubuntu)?

also see the Rrandom help file referencing 3.6

[sdtaylor]

using https://github.com/sdtaylor/phenology_gradients/blob/65dd3325ecdd88c578e35d3e8e690035879bf3f3/tests/testthat/test_simulation_tests.R

Office computer

• [ ] 3.4
• [ ] 3.5
• [x] 3.6 failed notes

Serenity

• [ ] 3.4
• [ ] 3.5
• [x] 3.6 failed notes

HPG

• [ ] 3.4
• [x] 3.5 passed notes
• [x] 3.6 failed notes

home computer

• [ ] 3.4
• [ ] 3.5
• [ ] 3.6

UF Lab windows computer

• [x] 3.4 failed notes
• [ ] 3.5
• [ ] 3.6

[sdtaylor]

UF Lab windows computer, R3.4

The tests fail but only cause the rounding is different past ~5 decimal places.

round(generate_random_spatial_gradient(x, y, seed = 6999), 7) not equal to c(0.9482594, 0.8344935, 0.5773765, 0.2222792, 0.0051186, 0.1434193).
5/6 mismatches (average diff: 1.92e-06)
[1] 0.94826 - 0.94826 ==  4.6e-06
[2] 0.83449 - 0.83449 == -1.0e-07
[4] 0.22228 - 0.22228 ==  1.7e-06
[5] 0.00512 - 0.00512 ==  5.0e-07
[6] 0.14342 - 0.14342 ==  2.7e-06

R version 3.4.1 (2017-06-30) -- "Single Candle" uses this code

[sdtaylor]

Office computer, R 3.6

Tests fail

test_simulation_tests.R:12: failure: random gradient generators are reproducible given a seed
round(generate_random_spatial_gradient(x, y, seed = 6999), 7) not equal to c(0.9482594, 0.8344935, 0.5773765, 0.2222792, 0.0051186, 0.1434193).
6/6 mismatches (average diff: 0.377)
[1] 0.163287 - 0.94826 == -0.78497
[2] 0.143696 - 0.83449 == -0.69080
[3] 0.099421 - 0.57738 == -0.47795
[4] 0.038276 - 0.22228 == -0.18400
[5] 0.000882 - 0.00512 == -0.00424
[6] 0.024703 - 0.14342 == -0.11872

R version 3.6.0 (2019-04-26) -- "Planting of a Tree" uses this code

[sdtaylor]

Serenity, R 3.6

tests fail, though differents are pretty close.

round(generate_random_spatial_gradient(x, y, seed = 6999), 7) not equal to c(0.9482594, 0.8344935, 0.5773765, 0.2222792, 0.0051186, 0.1434193).
5/6 mismatches (average diff: 1.79e-05)
[1] 0.94825 - 0.94826 == -1.10e-05
[3] 0.57740 - 0.57738 ==  2.07e-05
[4] 0.22228 - 0.22228 ==  3.00e-07
[5] 0.00512 - 0.00512 ==  8.00e-07
[6] 0.14348 - 0.14342 ==  5.69e-05

R version 3.6.0 (2019-04-26) -- "Planting of a Tree" uses this code

[sdtaylor]

HPG, R 3.5

tests pass

OK:       13
Failed:   0
Warnings: 7
Skipped:  0

R version 3.5.1 (2018-07-02) -- "Feather Spray" uses this code

[sdtaylor]

HPG, R 3.6

tests pass

test_simulation_tests.R:12: failure: random gradient generators are reproducible given a seed
round(generate_random_spatial_gradient(x, y, seed = 6999), 7) not equal to c(0.9482594, 0.8344935, 0.5773765, 0.2222792, 0.0051186, 0.1434193).
6/6 mismatches (average diff: 0.398)
[1] 0.765 - 0.94826 == -0.1835
[2] 0.768 - 0.83449 == -0.0666
[3] 0.772 - 0.57738 ==  0.1942
[4] 0.774 - 0.22228 ==  0.5521
[5] 0.775 - 0.00512 ==  0.7701
[6] 0.767 - 0.14342 ==  0.6240

R version 3.6.0 (2019-04-26) -- "Planting of a Tree" uses this code

[sdtaylor]

using https://github.com/sdtaylor/phenology_gradients/commit/9233f0ca90bd20d83edfd4b2dee4dd08d23529d1

this commit sets random seed parameters explicitly, which his what changes between version.

Office computer

• [ ] 3.4
• [ ] 3.5
• [x] 3.6 passed (note the new expected values were generated on this pc)

Serenity

• [ ] 3.4
• [ ] 3.5
• [x] 3.6 failed

HPG

• [ ] 3.4
• [ ] 3.5
• [x] 3.6 failed

home computer

• [ ] 3.4
• [ ] 3.5
• [ ] 3.6

UF Lab windows computer

• [ ] 3.4
• [ ] 3.5
• [ ] 3.6

[sdtaylor]

HPG 3.6

using fixed set.seed paramters

test_simulation_tests.R:12: failure: random gradient generators are reproducible given a seed
round(generate_random_spatial_gradient(x, y, seed = 6999), 7) not equal to c(0.2291873, 0.2781549, 0.3384481, 0.3830385, 0.3957438, 0.3074071).
6/6 mismatches (average diff: 0.448)
[1] 0.765 - 0.229 == 0.536
[2] 0.768 - 0.278 == 0.490
[3] 0.772 - 0.338 == 0.433
[4] 0.774 - 0.383 == 0.391
[5] 0.775 - 0.396 == 0.379
[6] 0.767 - 0.307 == 0.460

[sdtaylor]

Conclusion

Things are not reproducible across systems, even with setting the random number generator parameters to: set.seed(seed, kind = "Mersenne-Twister", normal.kind="Inversion",sample.kind = "Rejection")

Thus I've changed the analysis to reflect this. Before model building and error calculation were done in different scripts on different systems, thus the need for reproducible results across systems. Now I've changed this so everything is done on one system.

see model run changes in 3ffd83b674dd403497e8273b2d02ec6f2caa9f15 and test changes in 6a3213b589030e1142c6d900ba6f8e972f16909f