joseph-roitman / pytest-snapshot

A plugin for snapshot testing with pytest.
MIT License
114 stars 12 forks source link

Write dump file on failed test #58

Closed nickdelgrosso closed 12 months ago

nickdelgrosso commented 1 year ago

Hi,

This tool is really remarkably good! As in issue #54, I needed some kind of dump file to make data visualization comparisons in diff tools simpler on failed snapshot tests; this pull request just demonstrates a small change I made on my local installation that made the workflow very nice:

  1. Run the tests: pytest
  2. Use a diff tool to investigate failed tests (currently I just use PyCharm's diff tool, and compare the original file (e.g. myplot.png) with the observed file (e.g. myplot.dump.png), which has the same ending file extension to make it easier on diff tools.
  3. If approving, then update the snapshot with deletion to delete the dump files: pytest --snapshot-update --allow-snapshot-deletion

A caveat is file cleanup; I only found that the snapshot.assert_match_dir() function worked with --allow-snapshot-deletion, so a better implementation might be to do extra file deletion also on assert_match(). But using asset_match_dir() for now has been fine.

Anyway, I'm sure there is a better way to do this; I hope you'll consider including dump files as a feature for this really great tool!

Best wishes,

Nick