Open virtuoushub opened 2 years ago
There's no such ability as of now.
I'd love to see this as build in feature in playwright as well.
This would be a nice feature.. Seems like there was another issue requesting this: https://github.com/microsoft/playwright/issues/19597
One obvious difficulty is when running the tests on multiple CI machines (sharding).
Jest snapshot feature supports removing obsolete snapshots, might give some clues as to approach this, though I don't know if they support sharding. Also I think Jest puts all the snapshots for a test in a single file, and thus has full ownership of said file. PW on the other hand usually puts snapshots into a directory, so it would have to be a dedicated directory for PW to have authority to just delete stuff.
Simply deleting all snapshots and running tests (to regenerate snapshots) isn't really viable option for CI, and I'd like for CI to enforce no obsolete snapshots.
Having the option of PW deleting obsolete snapshots would be sufficient, since CI can detect the changes to branch, though would be nice also to have a flag which causes failure if there are excess snapshots. On CI it's nice to enforce strictness: all snapshots match and there's nothing extra.
If it helps others, the way I've dealt with this was to create a custom script to provide this functionality. The script works like this:
--update-snapshots
and setting the snapshot directory to be different than the default snapshots dir. Let's call the new snapshot directory the expected snapshots dir
and the default snapshots dir the actual snapshots dir
.actual snapshots dir
with the expected snapshots dir
actual snapshots dir
but don't exist on the expected snapshots dir
.Here is a gist that shows the script. Do read the first comment on the script that explains a bit more on how I'm using it.
Hope this helps others until we get this supported out of the box 🤞
I've decided to improve the initial version of the script provided on the gist from my previous comment.
I created the edumserrano/playwright-adventures repo to share some of the experiences I've had whilst using Playwright and one of them is a solution to deleting stale snapshots.
Check out the stale-screenshots-cleanup demo.
As a stopgap for this I added a custom reporter to fail CI when stale screenshots are present
When I first ran
playwright test -u
I expected it to remove any snapshots that were no longer related to any of my tests.Does playwright have the similar ability to
jest
where it can remove "stale"/outdated snapshots?I have tried out using
updateSnapshots
types"none" | "all" | "missing"
in my config, and nothing seemed to result in the desired behavior.Resources look at:
community resources
similar question #19597