review `server_threads`

[nilsnolde]

When setting this up on a client's machine, I just noticed that it started valhalla_service with 8 threads, even though I had server_threads = 4.

[danceb]

I don't know if this is related to this. But we had some issues getting the engine running on our system. We always got an error too many open files. The solution was to set the server_threads parameter especially within the configuration. It seems without it, the application is trying to run on every available core and not only on the amount which is needed.

[danceb]

It seems without it, the application is trying to run on every available core and not only on the amount which is needed.

Oh, according to the documentation on https://github.com/gis-ops/docker-valhalla#environment-variables that's the way it should work... But in this case the application should still run correctly and not giving the mentioned error, I think. So maybe there is some other issue related to this.

[nilsnolde]

Right, that’s the way it should work. But if you say 4 threads specifically, it should only use 4. however, in my case it was using 8.

The error you experienced is a luxury problem;) you have a machine with many threads and Valhalla exceeded your machine‘s „ulimit“. That‘s not a Valhalla problem, see https://github.com/gis-ops/docker-valhalla/issues/106.

[nilsnolde]

Though I still don’t understand how that can be if you’re using the tar file.. surely that guy didn’t have more than 65k threads😄actually it should only open 1 file per thread, mmap the tar. I’ll ask Kevin, seems I’m missing smth..

[iwpnd]

bumped my head with setting the ulimit correctly to avoid the too many open files but found that I've to pass the server_threads in the environment now specifically on docker run -e server_threads=4 [...] as opposed to using server_threads=4 docker run --rm [...] as I was doing it usually.

ulimit is pretty high when running a container and does not equal the hosts ulimit -n. the issue is that when not providing a limit to the server threads - and having a powerful machine with way too many cores - each thread is using a fair amount eventually exceeding the container limit.

When setting this up on a client's machine, I just noticed that it started valhalla_service with 8 threads, even though I had server_threads = 4.

I was able to replicate this. When setting up as above valhalla is spawning with 2 times the thread than defined - that is if I can trust the logs, showing that it loaded the tile extract 2 times.

[nilsnolde]

server_threads=4 docker run --rm [...]

this doesn't propagate the env var inside the container does it? to me it reads like this is just a env var for the process docker run which ends up spawning a container. so maybe it wasn't effective before?

usually ulimit should only have an impact when

• building a graph with elevation support, since the graph build will jump around the elevation tiles and open each one quite erratically
• having a huge graph and not using the tar file but the plain graph directory

When setting up as above valhalla is spawning with 2 times the thread than defined

yeah, when I read your comment, it dawned on me that valhalla_service is spawning quite a few threads internally for each "thread" we set. the logs we see are from the graphreader, 2 of those internal threads use one each, so it seems like it's using twice as many but it's actually all fine.

all in all, I'll close here, I don't think there's anything wrong with our logic here.