Closed Alvin-Robert-Lee closed 1 year ago
Seems like Xcode Cloud runners have Intel CPUs, which cause snapshots to diff+fail if you generated your snapshots on a machine running Apple Silicon.
how did you resolve this?
I tend to use the precision
and perceptualPrecision
parameters set to 0.98
. This allows us for some "wiggle room" in the snapshots and works ok
Thank you very much, i will try that! but did you have any other issue of accessing the reference images in Snapshot folder? For me, all tests pass locally but in xcode cloud all of them fails and there is no reason in logs.
I made a YouTube video exactly about that! https://www.youtube.com/watch?v=KPJL8vN3lAg
@krissaj @fespinoza
The solution is really to just record on the CI. That way snapshots are always being generated on identical devices. (You can still test locally against locally recorded snapshots.)
But you don't have to actually save the snapshots into the repository, take a look at https://screenshotbot.io (also open source): our plugin will handle all of this for you and you'll get notifications on your Pull Request when snapshots change, so you don't have to manually record snapshots each time.
We're also actively being used by large Android and iOS teams, it's especially useful when you want to scale your snapshots to 100s or 1000s of snapshots and don't want to force developers to manually re-record screenshots.
Has anyone experienced snapshots failing to match their reference on Xcode Cloud even though they match locally?
After fixing the issue where snapshots weren't being copied over, I still have some snapshots failing to match their reference image snapshot, even though they are the same visually. I've tried lowering the precision down to 0.98 and it still fails.
I've confirmed that the simulator iPhone and iOS version match my local environment. Xcode version also matches.
The only difference I could find is that the macOS version on Xcode Cloud is 13.4.1, and I'm on 13.4.