Image snapshot test is very unreliable

MarcellPerger1 / minecraft_clone_1

A minecraft clone using javascript & webgl

MIT License

2 stars 1 forks source link

Image snapshot test is very unreliable #183

Open MarcellPerger1 opened 10 months ago

MarcellPerger1 commented 10 months ago

Image snapshot test is very unreliable: ~15% difference and it's making CI fail.

[x] As a temporary fix, just increase the tolerance on the test to make CI work again
[ ] Fix the root cause (perhaps antialias=false would fix it)

MarcellPerger1 commented 10 months ago

As a temporary fix, just increase the tolerance on the test to make CI work again

Turns out changing the comparisonMethod to ssim made it work

MarcellPerger1 commented 5 months ago

Still seems to be a bit inconsistent, see https://github.com/MarcellPerger1/minecraft_clone_1/actions/runs/9618002711/job/26530972442

MarcellPerger1 commented 5 months ago

The problem isn't with my tests (probably), it's with my code or with WebGL. It puts the texture coords a bit off (and sometimes more than just a bit off) so the tests fail as the textures are off.

MarcellPerger1 commented 4 months ago

It seems to have weeks where it works well and weeks where it just doesn't want to work at all.