This is a test that runs fixedbasisMCMC for one iteration with 4 basis functions, over two days of data. It runs in about 20 seconds, so it should be useful for checking that minor changes to the code don't break existing functionality.
This test doesn't cover the run_hbmcmc script. The changes to hbmcmc.py are mostly due to black; there are a few changes to make it easier to choose where the quadtree basis is saved.
I added a test with more iterations, which is marked "slow". I set up pytest.ini to not run "slow" tests by default. The slow test uses a week of data, 50 basis functions, and 5000 iterations; it runs in about 7-8 minutes on my macbook.
In the future, I may add more "slow" tests that are marked "blue_pebble", which will only run on blue pebble, for tuning purposes. (Alternatively, this could be done with a separate script.)
For set-up, conftest.py creates an object in a temporary directory using some data files distributed with openghg_inversions. If this object store already exists and has data in it, then no data is standardised, which should speed up subsequent tests. (This is also a test_conftest.py file that checks if the right data is available.) Some data needed to run the inversion test isn't stored in the object store; this data is added to "bc_basis_functions" and "countries" by conftest when the tests are first run (if they don't exist, conftest creates these directories.)
To keep the openghg_inversions repo clean while running tests, I changed how quadtreebasis works: if "outputdir" is passed, then the basis is saved there, rather than in a temporary directory (which with the current code, would be created where the tests are run from). I also added some safe guards to the part of hbmcmc.py that deletes the temporary basis file. Saving the quadtree to the output folder is controlled by a flag in fixedbasisMCMC, so it doesn't change the behaviour of the code unless you ask it to.
This is a test that runs
fixedbasisMCMCfor one iteration with 4 basis functions, over two days of data. It runs in about 20 seconds, so it should be useful for checking that minor changes to the code don't break existing functionality.This test doesn't cover the
run_hbmcmcscript. The changes tohbmcmc.pyare mostly due to black; there are a few changes to make it easier to choose where the quadtree basis is saved.I added a test with more iterations, which is marked "slow". I set up pytest.ini to not run "slow" tests by default. The slow test uses a week of data, 50 basis functions, and 5000 iterations; it runs in about 7-8 minutes on my macbook.
In the future, I may add more "slow" tests that are marked "blue_pebble", which will only run on blue pebble, for tuning purposes. (Alternatively, this could be done with a separate script.)
For set-up,
conftest.pycreates an object in a temporary directory using some data files distributed withopenghg_inversions. If this object store already exists and has data in it, then no data is standardised, which should speed up subsequent tests. (This is also atest_conftest.pyfile that checks if the right data is available.) Some data needed to run the inversion test isn't stored in the object store; this data is added to "bc_basis_functions" and "countries" by conftest when the tests are first run (if they don't exist, conftest creates these directories.)To keep the
openghg_inversionsrepo clean while running tests, I changed howquadtreebasisworks: if "outputdir" is passed, then the basis is saved there, rather than in a temporary directory (which with the current code, would be created where the tests are run from). I also added some safe guards to the part ofhbmcmc.pythat deletes the temporary basis file. Saving the quadtree to the output folder is controlled by a flag infixedbasisMCMC, so it doesn't change the behaviour of the code unless you ask it to.Closes issue #8.