pointfreeco / swift-snapshot-testing

📸 Delightful Swift snapshot testing.
https://www.pointfree.co/episodes/ep41-a-tour-of-snapshot-testing
MIT License
3.8k stars 582 forks source link

Host application on framework testing target add additional top padding #474

Open arkulpaKonstantin opened 3 years ago

arkulpaKonstantin commented 3 years ago

Hello, I recently ran into the issue that the snapshots generated from a framework testing target add additional top padding. This only happens when I add a host application and not for all simulators. Some changes to our code sadly now require that we add the host application to our snapshot testing target.

To illustrate the issue I have created a sample project that only presents a red screen with a green background.

Simplified the assertion looks like this

assertSnapshot(
    matching: Color.red.background(Color.green.ignoresSafeArea()),
    as: .image(
        layout: .device(config: .iPhoneSe)
    )
)

As one can see from the provided snapshots below the top padding is different for setting the host application or not. To not be confused the device is set to iPhone SE (1st generation) and only the simulator is changed.

Without host application set on the iPhone 11 (14.5) simulator:

without_11_testView 1

With host application set on the iPhone 11 (14.5) simulator:

with_11_testView 1

There is no difference with the iPhone SE (2nd generation) simulator.

Without host application set on the iPhone SE (2nd generation) simulator:

without_SE_testView 1

With host application set on the iPhone SE (2nd generation) simulator:

with_SE_testView 1
sisoje commented 3 years ago

I tried with ios15 and its again different

lukeredpath commented 3 years ago

I'm unsure if this is related, but we've also noticed different snapshot output when running snapshots as part of an SPM test suite compared to running them in an app test target using a test host. In the process of modularising our app we started moving features and their tests into SPM modules and had minor snapshot failures as a result - a slight pixel shift downwards and also what appears to be minor color profile differences. I can only assume this has something to do with them no longer running in a test host.