CORE-GATECH-GROUP / serpent-tools

A suite of parsers designed to make interacting with SERPENT output files simple and flawless
http://serpent-tools.readthedocs.io/en/latest/
MIT License
52 stars 33 forks source link

Add infrastructure for testing plot methods #429

Closed drewejohnson closed 3 years ago

drewejohnson commented 3 years ago

This commit introduces some machinery that allows us to simply perform plot tests. A new directory tests/plot is added with a single test on our BWR spectrum detector, and the expected figure. This is more a test on how our CI system will handle plot comparisons, in particular the tolerances on the image comparisons may bite us.

The proposed workflow for adding a plot test is rather simple now. Tests should be added in the plot directory to a file that makes sense, and decorated with the compare_or_update_plot decorator. This performs a lot of work behind the scenes to make the test writing easy.

  1. Matplotlib defaults are restored
  2. Some Matplotlib test setup is performed using matplotlib.testing
  3. All existing figures are closed (before and after the test)
  4. Run the test function, which should generate a single figure
  5. If the --update CLI switch is given, this figure is saved to tests/plots/result_images/<test_name>.png and no further actions are taken
  6. If the --update CLI switch is not given, we save the generated figure to tests/plots/result_images/<test_name>-test.png and use matplotlib.testing.compare.compare_images to check the images

Using this decorator, the entire BWR spectrum test is four lines, two of which are the decorator and the function signature.

This will be very helpful for improving our coverage, since just running pytest doesn't test plot methods. Our notebooks do, but they don't check for consistency over time, just that a plot can be drawn

drewejohnson commented 3 years ago

Tests are failing for python 3.5 because it doesn't look like matplotlib supports saving to files using pathlib.Paths - https://travis-ci.com/github/CORE-GATECH-GROUP/serpent-tools/jobs/457429722#L1071-L1131

Python 3.5 is already past end-of-life and I've proposed to drop it in #425, probably in the next major release