vitest-dev / vitest

Next generation testing framework powered by Vite.
https://vitest.dev
MIT License
12.8k stars 1.15k forks source link

Interactive snapshot update/create mode #2229

Open mrspartak opened 1 year ago

mrspartak commented 1 year ago

Clear and concise description of the problem

We are testing media assets, by matching their md5 basically. So we have a suit, that basically calls the same render function but in different contexts (projects, parameters). Basically, the output must be validated before confirming, that this md5 should be a snapshot. Jest has an interactive mode, but async describe is not a thing anymore, so we tried to switch to Vitest.

Suggested solution

  1. Maybe do not create snapshots by default (I know, that env CI=1 solves it)
  2. Have interactive update/create mode, that does 1 test at a time and asks to validate output

Alternative

I could implement it myself but have no way to interrupt runner and it just times out. Also, libraries such as enquirer do not work during the run.

Additional context

No response

Validations

sheremet-va commented 1 year ago

I could implement it myself but have no way to interrupt runner and it just times out. Also, libraries such as enquirer do not work during the run.

I think we can just run all the tests, and if there are snapshot mismatches, instead of an error we can generate new ones and show it in the terminal, like jest does, so user can confirm its content. After the change is confirmed, we mark it as resolved.

mrspartak commented 1 year ago

Yes, basically interrupt the runner until the action in the console is resolved (confirm or not the snapshot).

mrspartak commented 1 year ago

I just thought, would be cool, if snapshotMatch method would return if a snapshot is updated. We create media and would like to store confirmed in the cloud so we can also compare them later visually in case of a snapshot (which basically md5) mismatch.