Closed wipfli closed 2 years ago
Searching for render-test
across the codebase shows something like 30 results (maplibre-gl-js/ folder excluded).
I think it is possible to run the render tests in linux with something like this:
git clone --recurse-submodules -j8 https://github.com/maplibre/maplibre-gl-native.git
cd maplibre-gl-native
# build stuff
cmake . -B build -G Ninja -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER=gcc-10 -DCMAKE_CXX_COMPILER=g++-10
cmake --build build -j $(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null)
# run render test
./build/mbgl-render-test-runner --manifestPath metrics/linux-clang8-release-style.json &> render-test.log
# remove color stuff from log file
sed -i 's/\x1b\[[0-9;]*m//g' render-test.log
The following 11 tests fail currently:
* failed render-tests/symbol-placement/line-overscaled
* failed render-tests/symbol-visibility/visible
* failed render-tests/text-variable-anchor/rotated-offset
* failed render-tests/text-variable-anchor/all-anchors-tile-map-mode
* failed render-tests/icon-text-fit/enlargen-both
* failed query-tests/circle-radius/tile-boundary
* failed query-tests/fill-features-in/default
* failed query-tests/invisible-features/zero-opacity
* failed query-tests/edge-cases/box-cutting-antimeridian-z1
* failed query-tests/edge-cases/box-cutting-antimeridian-z0
* failed query-tests/edge-cases/null-island
Here are the 5 failing render tests:
symbol-placement/line-overscaled:
symbol-visibility/visible:
text-variable-anchor/rotated-offset:
text-variable-anchor/all-anchors-tile-map-mode:
icon-text-fit/enlargen-both:
Ah, beautiful. By reverting the fix of #285, i.e., with
if (maxLineHeight != lineHeight) {
shiftY = -blockHeight * verticalAlign - Shaping::yOffset;
} else {
- shiftY = (-verticalAlign * static_cast<float>(lineCount) + 0.5f) * lineHeight;
+ shiftY = (-verticalAlign * static_cast<float>(lineCount + 0.5)) * lineHeight;
}
we get 298 failing tests...
Here is one example:
At commit cf41e113f9225379bc5d32002a3802b4d2d3ad94, i.e., right before #270, only one test failed. It was icon-text-fit/enlargen-both
.
At commit 32ed70c95d734590b3e68cd4595a2806fd13c389, i.e., right after #270 many tests fail. If I then add the fix from #285, the 11 tests fail which currently fail on main
. I guess this means that #270 introduced changes that made 10 tests fail.
At 340f3d6cd (Oct 2021), 71 tests fail icon-text-fit/enlargen-both. That was before the boost update.
Oops 😊
But now knowing how to run these tests, we may be able to find the remaining typos that I produced.
There is actually a nice website generated with all the test results. Here is an example with the failing tests from #339:
render-tests/icon-text-fit/enlargen-both
works for me on macOS. It may be interesting to know if this is something platfom-specific.
Also why are some tests ignored?
Ah interesting... Why tests are ignored I don't know. But have have that in MapLibre GL JS as well...
closed by #357
MapLibre GL JS has render tests. They consist of a
style.json
file and anexpected.png
image. GL JS renders the style and compares the generatedactual.png
withexpected.png
. If the pixel difference is too large, the test fails. Here is an example:https://github.com/maplibre/maplibre-gl-js/tree/main/test/integration/render/tests/circle-color/default
MapLibre GL Native uses the exact same render tests as GL JS. There are test apps for iOS, Android, and linux which are there to run the render tests.
The android makefile has a run command:
https://github.com/maplibre/maplibre-gl-native/blob/5b3d936521040149e343f81ed4bd2ce24b75e690/platform/android/Makefile#L174-L191
I would like to find out if I can run the render tests manually, and if yes, how we can run them in CI...