fabern / LWFBrook90.jl

Istope-enabled implementation of the LWF-BROOK90 hydrological model in Julia
GNU General Public License v3.0
11 stars 1 forks source link

test: Use correct RNG for hardcoded values in regression test #29

Closed fabern closed 2 years ago

fabern commented 2 years ago

Intent: Using the BEA simulation as regression test.

Problem: Tests with hardcoded values do not pass successfully on both, local MacBook Pro and remote CI system (GitHub Actions). The floating point comparison is not accurate enough.

Issues could be linked to DifferentialEquations.jl using some random numbers for time stepping. Though, according to https://discourse.julialang.org/t/tests-failing-on-linux-in-travis-ci/19602/2 the @testset macro should initialize the random number seed automatically. It could also be some other issue. Indeed, the 7th to 10th state variables of example_result["solution"].u[10][idx_of_state_variables] show rather large differences of up to 0.60.

codecov-commenter commented 2 years ago

Codecov Report

Merging #29 (57c3a2b) into develop (66abdca) will not change coverage. The diff coverage is n/a.

:exclamation: Current head 57c3a2b differs from pull request most recent head cd8e074. Consider uploading reports for the commit cd8e074 to get more accurate results

Impacted file tree graph

@@           Coverage Diff            @@
##           develop      #29   +/-   ##
========================================
  Coverage    79.51%   79.51%           
========================================
  Files           15       15           
  Lines         1455     1455           
========================================
  Hits          1157     1157           
  Misses         298      298           

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 66abdca...cd8e074. Read the comment docs.

fabern commented 2 years ago

Idea was to address this in f75b67bd944ac18b87e1aa8c570ff3ca7459b3f0 by using the *.jld2-files. However the tests generated locally still do not work on the CI system -> see pull request #33