linebender / xilem

An experimental Rust native UI framework
Apache License 2.0
3.74k stars 118 forks source link

Improve snapshot tests #363

Open PoignardAzur opened 1 year ago

PoignardAzur commented 1 year ago

Right now the assert_render_snapshot! macro and the underlying code are held together with duck-tape.

Some possible improvements:

PoignardAzur commented 7 months ago

(Copied from some notes I took during the Winit port:)

The basic idea with snapshot tests is simple: we create a widget tree that we want to test; we lay them out and render them to a bitmap; then we save that bitmap to the repository. In future tests, we do the process again and check that the bitmap hasn't changed. If it has, it likely indicates either a bug or that our rendering code changed.

That's great in theory, but in practice these tests tend to be flaky. There are several things that can introduce false negatives in CI:

To fix snapshot tests, we need be able to reliably avoid these issues.