Rewrite tests to make use of `pytest`

[tztsai]

This PR rewrites the testing approach in SPINacc. Previously, tests were written to execute at the end of a run. There are two problems:

• This lengthens the program execution time
• Tests have only been written for a single version of ORCHIDEE (corresponding to the data on Zenodo).

Instead, this PR introduces pytest, which allows users to run tests after a step of the pipeline has completed. To utilise pytest all previous tests have been renamed to test_task*.py and all functions now have a test prefix.

To summarise:

• Renamed config.py to conftest.py (apytest recognised configuration file)
• Renamed all previous tests files to test_task*.py and all test functions renamed with a prefix test_
• Pytest markers are used to parametrize the input variables such as the names of the files to be compared.
• Constant variables used in the testing are defined as pytest fixtures,.
• Commonly used functions that compare the contents of the reference file and the result file have been moved into utils.py.
• The utils.py includes a recursive compare function, as this was necessary to check arrays of arrays.

Execution

To run the tests, use the command pytest tests/.

Automatic testing

The tests rely on a job being completed before execution as output data needs to be there for a comparison to happen. For automated testing (regression), this presents a problem. We have therefore shown how it may be possible to use github actions to run the job in advance and then execute a single test.

The automated job needs the following data to execute a test.

• reference data for comparison (zenodo)
• simulation input data.

The zenodo data is approximately 1.1GB. We cannot run SPINacc from scratch because the data dependencies are just too large. This is explained more here

Testing is steered using a .github actions file. .github/workflows/python.yml, and assumes that the initialisation step has already been executed. Therefore auxil.npy and packdata.npy are already there.

It does the following:

• Clones reference data from https://github.com/ma595/SPINacc-data.git including:
  • auxil.npy (metadata) and packdata.npy
  • vlad_data/vlad_data
  • Does some sed to change the execution paths and the test to test_clustering. Equally, we could commit a file and move this to the appropriate location. (Doesn't have to be DEF_Trunk, for example).
  • Change paths in varlist.json to the appropriate path corresponding to the directory of the cloned vlad_data.
  • Runsmain.py
  • pytest then executes tests/test_task1.py only.

Outstanding issues

In order to merge the following issues should be addressed:

• [x] Hardcoded file path in tests directory should be reconsidered.
• [x] Check that the computed and reference outputs correspond
• [ ] We could encapsulate the above steps in pytest (executing the code shouldn't be separate).
• [ ] The git history needs to be resolved (rebase) to reflect changes to reformating of data to netcdf and other changes.
• [ ] The runtime (reproducibility) tests fail now that the files have been renamed.
• [ ] Determine to what extent we should continue this PR given the data inputs necessary. The use of github currently may not be the best approach - and is only really for illustration.
  • Consider reducing the dataset size and hosting on zenodo.

Closes #37 and #40.

[ma595]

All tests pass locally for me, except for step 3 which we don't care about at the moment.

[TomMelt]

Discussed idea of generating idealized test case for integration test. Not necessarily physically meaningful but something that uses the code in a realistic manner.

@dsgoll123 will make a new small training set for new version ORCHIDEE