search

LLNL / thicket

https://thicket.readthedocs.io
MIT License
14 stars 9 forks source link

Run Unit Tests for Different Parameters #182

Open michaelmckinsey1 opened 3 months ago

michaelmckinsey1 commented 3 months ago

This PR proposes to expand running the unit tests for (1) Thickets with intersection trees (default is union) and (2) Thickets without filling the performance data (default is filling the performance data). Therefore, testing for each combination would run all the unit tests for 4 types of Thickets:

  1. Union+Filled
  2. Union+NonFilled
  3. Intersection+Filled
  4. Intersection+NonFilled

Example of using parametrized fixtures from pytest docs

When tests fail, we can see which configuration of parameters failed:

FAILED thicket/tests/test_tree.py::test_indices[Intersection-FillPerfdata] - Key...
FAILED thicket/tests/test_tree.py::test_indices[Union-NoFillPerfdata] - KeyError: ...

We can run single tests for a single set of parameters by specifying them to pytest

$ python -m pytest thicket/tests/test_tree.py::test_indices[Union-NoFillPerfdata]

============================= test session starts =============================
platform win32 -- Python 3.11.7, pytest-8.0.0, pluggy-1.4.0
rootdir: C:\Users\Micha\Documents\Github\thicket
configfile: pytest.ini
collected 1 item

thicket\tests\test_tree.py F                                             [100%]
michaelmckinsey1 commented 2 months ago

Depends on ~#193~

michaelmckinsey1 commented 2 months ago

depends on ~#181~

michaelmckinsey1 commented 2 months ago

@ilumsden Tests that use the thicket_axis_columns fixture are already parametrized, since thicket_axis_columns is parametrized itself. Unlike the other fixtures, which are lists of files, thicket_axis_columns and stats_thicket_axis_columns create the Thickets in the fixtures.

So running python -m pytest thicket/tests/test_concat_thickets.py::test_filter_stats_concat_thickets_columns results in:

0.44s call     thicket/tests/test_concat_thickets.py::test_filter_stats_concat_thickets_columns[Intersection-FillPerfdata]
0.41s call     thicket/tests/test_concat_thickets.py::test_filter_stats_concat_thickets_columns[Intersection-NoFillPerfdata]
0.33s call     thicket/tests/test_concat_thickets.py::test_filter_stats_concat_thickets_columns[Union-FillPerfdata]
0.33s call     thicket/tests/test_concat_thickets.py::test_filter_stats_concat_thickets_columns[Union-NoFillPerfdata]