search

syrupy-project / syrupy

:pancakes: The sweeter pytest snapshot plugin
https://syrupy-project.github.io/syrupy/
Apache License 2.0
527 stars 36 forks source link

Add functionality for Expect Testing #825

Open harryjulian opened 11 months ago

harryjulian commented 11 months ago

Is your feature request related to a problem? Please describe.

I like using syrupy for snapshot testing across my projects, but I'd be interested in expect testing, which is outlined in this article on the Jane Street tech blog.

The main difference between regular snapshot testing and expect testing is the interactive component - the developer is shown either the test result, or a diff, in the console and can save the result with a keybinding if they deem the result correct and appropriate.

Sometimes when I'm using snapshot testing I'd like to actually know what snapshots look like to quickly manually verify them. I also imagine this could be useful for a sort of TDD utilising snapshotting, where you write your tests a priori and then iterably verify the outputs until they're correct.

Describe the solution you'd like

I'd like to implement a means of conducting expect testing in syrupy. This just means expanding the framework to, if expect testing is somehow toggled, add an additional step at the end of testing where the user is presented with an interactive console session to accept/reject test outputs if there are either diffs or not results.

To keep this sort of testing as a separate entity, I'd probably think about implementing a new fixture - expect - through which to manage the process. I'd give more thought to the implementation if the feature is of interest. If so, I'd be happy to collaboratively work on implementing it.

noahnu commented 11 months ago

This is an interesting idea. Have you seen another pytest plugin that adds some interactivity?

harryjulian commented 11 months ago

I've not used any plugins that use interactivity before, but I've made a simple plugin here that adds a very basic/ugly version of the post-test accept/reject functionality.

thijsmie commented 2 months ago

I'd also be interested in this, since right now I'm effectively running the whole testsuite twice in different modi, once to get the failures, then inspecting the failures whether the update is correct and then with snapshot-update. Since I have all check tools wrapped up in a makefile of sorts I would love to run make all --fix which allows me to interactively update the snapshots and then in CI make all which rejects updates as normal.