min/maxRendererPoolSizes in config have no effect

[geoneutrino]

I'm using docker, tileserver version 4.6.5 and am serving vector and raster Investigating slow raster performance i saw in htop that max 4 cpu cores are used. Changing min/maxRendererPoolSizes don't have an effect on this, no matter what i enter as value. Changing other options like maxScaleFactor increase from 3 to 4 works, only the renderers seem to use defaults (also changing docker runtime like "docker update --cpus 8 containerid" doesn't do any difference) It seems that the config options doesn't get to https://github.com/maptiler/tileserver-gl/blob/master/src/serve_rendered.js#L1604C36-L1604C56

What am i missing ?

my config.json { "options": { "paths": { "fonts": "fonts", "styles": "styles", "sprites": "sprites", "mbtiles": "" }, "maxScaleFactor": 3, "maxSize": 2048, "minRendererPoolSizes": [16,8,8], "maxRendererPoolSizes": [32,16,8], "tileMargin": 30 }, "styles": { ...

[geoneutrino]

Update: same effect when using source direct without docker console.logging the values in serve_rendered i see the pool is created with the adjusted value But in static rendering complex raster tiles i still see only 4 cpu cores used in htop

[acalcutt]

I tested the current code a bit yesterday and like you said it does seem to pass those values on... I'm not sure why it would still be limited to 4 cpu though.

[acalcutt]

Could it be related to UV_THREADPOOL_SIZE https://github.com/maptiler/tileserver-gl/blob/master/src/server.js#L5 you think?

[geoneutrino]

i have narrowed it down to maplibre native https://github.com/maplibre/maplibre-native/blob/main/ARCHITECTURE.md => "This work happens on "worker threads" that are spawned by the main thread, four per Style object. The Style and its associated objects handle dispatching tasks to the workers, typically on a tile-by-tile basis."

so, ok, our data and style was not well optimized and with changes to both we could improve performance notably

From a DevOps perspective then the "issue" remaining is a bit the question how to optimise the system for high load scenarios to fit your hardware as we seem to have min/maxRenderPoolSizes created by tileserver (not having the impact i expected first) Node.js UV_THREADPOOL_SIZE alsways set to cpu * 1.5 regardless of RenderPoolSizes and maplibre-natives 4 threads limit of style regardless of hardware ressources

[geoneutrino]

at least we could add some hint this into documentation

[acalcutt]

Looking for information on UV_THREADPOOL_SIZE I found this 2020 thread https://github.com/maptiler/tileserver-gl/issues/489 that indicates maybe it isn't working and setting it externally or moving this code to main.js could help

[acalcutt]

Did you ever get a chance to see if setting UV_THREADPOOL_SIZE environment variable elsewhere helped at all?

[geoneutrino]

Sorry for the late reply, as we changed data last year to optimize all it was harder to reproduce. But with release 4.10s Raster 512px its easier to test the effect as those are slower than the 256px. Long story short

1. the effect is still the same after changing code. So its the "four threads per style" of maplibre-native i mentioned in https://github.com/maptiler/tileserver-gl/issues/1072#issuecomment-1843141655 Not tileserver-gl issue but we can mention it in documentation

2. as far as i understand @vcabbage in https://github.com/maptiler/tileserver-gl/issues/489 moving the code from server.js to main.js should still be done to work correctly (and also check if already set before overwritung as he mentions) Especially when hosting multiple map styles then maplibre-nativ would create 4 threads per style