search

maplibre / maplibre-gl-js

MapLibre GL JS - Interactive vector tile maps in the browser
https://maplibre.org/maplibre-gl-js/docs/
Other
6.65k stars 715 forks source link

Render test passes even though actual image is different from expected. #4862

Open NathanMOlson opened 3 weeks ago

NathanMOlson commented 3 weeks ago

The terrain/fog render test passes, even though the actual image is different from the expected image.

Expected: expected

Actual: actual

Actual (drawn over a red background to demonstrate the transparent "hole" near the horizon): actual_red_background

Computed Diff: diff

HarelM commented 3 weeks ago

The render test are using an external library to compare images: pixelmatch with some configuration. The configuration is there to prevent false positive test failures. Feel free to investigate what's causing this...

NathanMOlson commented 3 weeks ago

Feel free to investigate what's causing this...

The root cause of this bug is that pixelmatch does not detect any difference between white and transparent pixels: https://github.com/mapbox/pixelmatch/issues/26

HarelM commented 3 weeks ago

Ahh, in that case we should make sure either not to use white or use a solid non-white background. Another approach we can try is contribute the fix to picelmatch, but I don't have high hopes for this route.

NathanMOlson commented 3 weeks ago

I submitted a PR to pixelmatch to fix this issue: https://github.com/mapbox/pixelmatch/pull/142. I think it would be a good idea to incorporate this fix, as it may affect other render tests in our test suite.

HarelM commented 3 weeks ago

Worst case we can patch-package it I guess. Let's wait a few days and see where this lands.