search

systemed / tilemaker

Make OpenStreetMap vector tiles without the stack
https://tilemaker.org/
Other
1.46k stars 230 forks source link

Introducing a test framework #364

Open guillaumerose opened 2 years ago

guillaumerose commented 2 years ago

I am trying this project and it works well for me. It is far more convenient and efficient. I can see some differences between the output of tilemaker and the output of openmaptiles.

Would that make sense to write some tests where we pass a .osm.pbf with few roads, etc. and assert the output is correct ? I see the CI generates a small area but there are no expectations. Should we add some ?

pnorman commented 2 years ago

Are you wanting to test tilemaker, or the Lua scripts that come with it?

guillaumerose commented 2 years ago

I think we should start with an end-to-end test to cover most of the features and ensure no regressions when a PR comes in.

systemed commented 2 years ago

I like the principle. Possibly a pipeline like .pbf -> tilemaker -> .mvt -> .geojson, and then we parse the geojson to make sure the features are correct. Testing the tile filesize would be useful too. Potentially quite a big undertaking though!

kleunen commented 2 years ago

The project would definitly benefit from testing. Even if the conversion fails and generates no output, the CI passes. I have seen the tilemaker executable crashing in CI, but CI says build is complete. So even checking if the output file exists and is not empty, would be an improvement

systemed commented 8 months ago

We're now using minunit for some basic tests thanks to @cldellow.

Ideally I'd like to test that output geometries render correctly (for example, generate the Hebrides at several zoom levels and compare the results to reference images). But that would probably require a headless browser and be a pretty big undertaking.