Advanced Model validation tests

[rburghol]

Overview

Develop specific performance (accuracy, and/or efficiency) testing of hsp2 model runs. Extends work #135

• base is test10 https://github.com/respec/HSPsquared/tree/master/tests/test10/HSP2results
• existing tests verify that:
  • hsp2 import completes correctly
  • hsp2 run completes without error
• Test should verify:
  • Assert that run data in a given output in test10.h5 is simulated correctly, like
    • Mean, median, min Q??
    • ?

Tasks:

• [x] Develop accuracy tests for test10 - basic testing now compares all simulation variables from HSPF run to hsp2 run, during each timestep of the simulation: https://github.com/respec/HSPsquared/blob/develop/tests/test_regression.py
• [x] (will not do for the time being) Develop working install for environment_dev.yml (see https://github.com/respec/HSPsquared/blob/dependency_options/environment_dev.yml)

Code

Install pytest etc.

pip install -e .[dev] # make sure test dependencies are installed
pip install pytest-xdist # allow running tests in parallel
pip install pytest-cov # permit more verbose/detailed tests

Compare Outputs from test10.uci and test10specl.uci

Basic testing in HSPsquared/tests/ (from @PaulDudaRESPEC @austinorr)

• Run code without numba: NUMBA_DISABLE_JIT=1 hsp2 run test10specl.h5
• Run quick test: pytest -k test_case
• Run comparison btwn hsp2 and hspf: pytest -k test_case -n 2
• Show every line of code not hit (requires pytest-cov): NUMBA_DISABLE_JIT=1 pytest -k test_case --cov --cov-branch --cov-report term-missing

Very simple pytest

• when a test file is loaded, functions in that file prefixed with test_ will be automatically executed.

  
  import pytest
  import os

def test_h5_file_exists(): assert os.path.exists('test10.h5')

This shows how how to break the test

def test_that_should_fail(): assert os.path.exists('nonexistent_h5_file.h5')