Possible Testing Schemes

[jukent]

Viz doesn't have many tests because the products are visually as expected or not, but how does matplotlib or cartopy have a testing scheme?

[jukent]

I think the first testing scheme we want to include is independent viz examples that build. First, let's take inventory of how many tests we need to add at minimum.

In util.py there are 16 different methods. In taylor.py there are 10 (external). I'll start with util.

Should these .py files formatted as in the geocat examples gallery or should they be notebooks? Removing upstream examples action, need to add infrastructure to build and add examples gallery to documentation (as done in comp).

[jukent]

Add example script to each function (divvy these up?):

• [x] add_height_from_pressure_axis() - JK
• [x] add_lat_lon_gridlines() - JK
• [x] add_lat_lon_ticklabels() - JK
• [x] geocat.viz.taylor.TaylorDiagram() - JK
• [x] set_tick_direction_spine_visibility() - JK
• [x] add_right_hand_axis() - JK
• [x] add_major_minor_ticks() - JK
• [x] set_titles_and_labels() - KF
• [x] set_axes_limits_and_ticks() - KF
• [ ] truncate_colormap() - KF
• [ ] xr_add_cyclic_longitudes() - KF
• [ ] set_map_boundary() - KF
• [x] find_local_extrema() - JK
• [x] plot_contour_labels() - JK
• [x] plot_extrema_labels() - JK
• [ ] set_vector_density() - KF
• [x] get_skewt_vars() - JK

Steps:

1. Search geocat-examples for simple example of fx, par down even more
2. Make .ipynb file for example
3. Save thumbnail
4. Add to docs/gallery.yml
5. Add link to fx docstring under examples (before examples gallery examples) EX: ‘See this example notebook: :doc:../../examples/add_height_from_pressure_axis. More in-depth plotting examples that utilize this function are in the GeoCAT-Examples Gallery.’
6. Add to examples.rst

[jukent]

Talked to @dcamron from Metpy about what they do. He directed me to their pizel-by-pixel testing using pytest-mpl, but indicated that they were unhappy with this scheme and didn't want us to get tangled up in it as they are investigating a better solution themselves. Still posting links here for now and will try to set up a meeting to learn about what it is they do not like about these tests.

https://github.com/Unidata/MetPy/blob/main/tests/plots/test_skewt.py https://github.com/Unidata/MetPy/tree/main/tests/plots/baseline

[kafitzgerald]

Maybe this belongs in a different issue, but I just noticed we're not actually building the docs in the nightly CI (which just failed, but because of a connectivity fluke). It would be nice to have some sort of tests run there.

We might also want to think about something a little more robust since a few things just got caught over on geocat-examples that probably should be failing over here as well (i.e. the matplotlib 3.8 API change and the problem with sphinx-book-theme and sphinx 7).

[jukent]

@kafitzgerald What is the status of truncate_colormap(), xr_add_cyclic_longitudes(), and set_map_boundary() examples?

[kafitzgerald]

All four w/ my initials are in progress. I got a bit hung up on how best to display the colormaps, but can get back to those this week and get them in.

[jukent]

I'll work on the skewT one this week.

[jukent]

Links to matplotlib's testing scheme, from our drop in on the new developers meeting:

• https://github.com/matplotlib/pytest-mpl?tab=readme-ov-file
• https://matplotlib.org/devdocs/api/testing_api.html#module-matplotlib.testing

They do test by comparing images. The most common unexpected failure trigger comes from the fonts changing They recently changed their testing scheme, and seems like they're still figuring it out to some extent

Seems like we could have a separate repo to not swamp this one down (or have it excluded from the __init___.py somehow that has images for every keyword argument entered into our functions that we can think of.