Add precision and tolerance apis

[fruitcoder]

The missing tolerance was mentioned here #15 but no PR was added yet. I'll leave this here while I'm experimenting with the implementation to fix failing snapshots on my CI.

[mattprowse-xero]

I'm in the process of adding AccessibilitySnapshot to a project I'm working on, and this is our last remaining blocker. I noticed #64 from a bit over a year ago: is there anything I can do to help get this over the line? More than happy to create a new PR if needed.

[fruitcoder]

We're still using our fork. Would be great to get this merged ir any info what would contribute to it being merged

[NickEntin]

Hey! Sorry, this one fell through my GitHub queue.

We've been exploring snapshot precision/tolerance APIs internally for snapshots in general. Reducing precision is unfortunately necessary when working across multiple types of machines, but we've seen a lot of cases where it's easy to let non-trivial regressions slip through with reasonable precision levels. I'm hoping we can find some alternative comparison methods that resolve the issue while reducing the possibility of letting through regressions.

cc @jhneves

[NickEntin]

@fruitcoder Does using per-pixel tolerance alone (without overall tolerance) fix your CI issues? I think that one is a bit safer than overall tolerance.

[fruitcoder]

Are you talking about the perceptualPrecision that's mentioned in the new swift-snapshit-testing 1.10.0 release or was this something I could have used before. If so, can you point me in the right direction? 😅

[NickEntin]

Ahh you're using SnapshotTesting. Per-pixel tolerance is only supported out of box in iOSSnapshotTestCase right now (see overallTolerance vs perPixelTolerance here).

Perceptual precision is very similar to per-pixel precision, just using a different definition for how "similar" two pixels are. It potentially could be more reliable than standard per-pixel precision, but I haven't had a chance to run it through our test suite yet.

[NickEntin]

Hey @fruitcoder, just following up here - did you get a chance to test whether per-pixel/perceptual precision is sufficient to fix your CI issues?

[fruitcoder]

Hey @NickEntin! Is there anything I can test here? I'm using snapshot testing and still can't find a way to get an image strategy that uses any kind of tolerance 🤔

[NickEntin]

@fruitcoder Try this branch. That should give you a parameter for .accessibilityImage(perceptualPrecision: ...). Try something like 0.995 to start.