Testing

[mhvwerts]

We should set up some simple automated testing infrastructure instead of testing by manually executing the examples, Notebooks and test scripts.

The test could involve

• running all examples and testing if they still give the same correct results
• running all Notebooks and seeing if they still give the same output
• running additional tests such as those in the tests folder and fvtool_test.py
• running the doctests that are in the docstrings

pytest may be the suitable tool. I used it once in the past, and even though the documentation looks daunting, I believe it wasn't too hard to set up basic testing, which with hindsight prevented some disastrous code changes in that project, avoided incorrect calculations and arguably saved time.

Some additional info:

• pytest for running doctest
• pytest and Notebooks

[simulkade]

It's a good idea @mhvwerts. Let me do some reading on pytest first. I did consider writing tests but got occupied with other stuff. Now that I'm switching to PyFVTool for some other projects, it must be easier to spend some time on it. I know that the current validation tests are working fine with unittest, but I'm not sure whether it is the most efficient way of managing tests. I'll do some research and experiments and come back to you.

[mhvwerts]

I did notice the tests folder, but did not try to run unittest. Apparently pytest will run unittest tests 'out-of-the-box': https://docs.pytest.org/en/7.4.x/how-to/unittest.html

I did some initial experimentation with pytest-notebook alone, without integrating it with a pytest. It seems to work, but I will need to get back to it later.

[mhvwerts]

To do:

• [x] define a task list for finishing the set-up of automated testing

[mhvwerts]

To do:

• [x] fix existing tests such that pytest completes without failures
• [x] check that pytest sees all currently available tests (explicit tests, but also implicit tests)
• [x] add a mechanism for obtaining benchmarking test results from the example scripts
• [ ] add additional tests to increase coverage
• [x] add testing instructions to documentation (installing pytest, pytest_notebook etc. - see pytest.ini)

[mhvwerts]

pytest is now functional (#22 )

See comments in pytest.ini for indications on setting up testing environment.