maptiler / tileserver-gl

Vector and raster maps with GL styles. Server side rendering by MapLibre GL Native. Map tile server for MapLibre GL JS, Android, iOS, Leaflet, OpenLayers, GIS via WMTS, etc.
https://tileserver.readthedocs.io/en/latest/
Other
2.24k stars 639 forks source link

Random crashes when rasterizing tiles #1411

Open georgbachmann opened 6 days ago

georgbachmann commented 6 days ago

We are using tileserver-gl to serve our maps. We have apps that only use vector tiles and that is NO PROBLEM at all. We separated all incoming requests already and noticed that serving pure vector data does not cause any problems. But for our website we still use raster tiles and that seems to cause problems after a couple of hours. We get consistent crashes and we think (not know) that it has something todo with memory consumption?!? At least memory seems to spike before crashes happen. The machines would have plenty though (50GB) but it's already crashing way before that. Currently we use version 4.11.1 and also tried the latest 5.0.0 to test and still the same problem. It's running on linux machines in a docker-container. (maptiler/tileserver-gl:v4.11.1)

We can also not give more information about what causes the issue, cause we don't know it and the logs of our docker-container don't say much.

So my first question would be: how to provide information to narrow down the problem?

acalcutt commented 3 days ago

Does this seem similar to https://github.com/maptiler/tileserver-gl/issues/1236 . Possibly try 4.5.1 and see if the issue happens in that version just so we can get an idea of when this started.

In docker are you able to see how much memory is allocated and in use? My understanding is y default all memory is available, but maybe that isn't the case?

I run 5.0.0 without the docker here on Ubuntu 22.04 and haven't noticed any issues, but some people have reported similar issues when using the docker.