Server fails to handle requests with large number of locations

[janusw]

I am trying to retrieve elevations in large batches.

Since max_locations_per_request by default is not very large (100), I increased it to 1000 on my server. This helped a bit, but actually I could not go all the way up to 1000 locations. Up to 400 or so it works well, but then I start getting errors like: http.client.RemoteDisconnected: Remote end closed connection without response

Unfortunately this error message is not very useful. It occurs in a python script that I use for testing, and these requests go to the server through a load balancer.

The first thing I tried was increasing the timeout in the python script, and even disabling it completely, but that did not help.

Then I went on to debug it more locally, without going through the LB, and finally got responses like: 414 Request-URI Too Large. The errors seem to come from nginx, so apparently this is an nginx limitation.

[janusw]

A quick web search reveals that nginx has a parameter called large_client_header_buffers, see: http://nginx.org/en/docs/http/ngx_http_core_module.html#large_client_header_buffers

This is 8k by default and limits the maximum length of a request. Increasing it (in nginx.conf) should fix my problem, I assume (haven't tried it, but will do soon).

[janusw]

So, yeah, increasing large_client_header_buffers indeed seems to help, at least partially. Here is what I do:

diff --git a/docker/nginx.conf b/docker/nginx.conf
index f3776a7..6ed746a 100644
--- a/docker/nginx.conf
+++ b/docker/nginx.conf
@@ -10,10 +10,12 @@ server {

     server_tokens off;

+    large_client_header_buffers 4 32k;
+
     location / {
         include uwsgi_params;
         uwsgi_pass unix:/tmp/uwsgi.sock;
         uwsgi_read_timeout 10s;
         uwsgi_send_timeout 10s;
     }
-}
\ No newline at end of file
+}

This does get rid of the 414 when debugging locally, but from the outside I still get http.client.RemoteDisconnected. So I assume there is an additional limitation with the load balancer or firewall :/

[ajnisbet]

This should be fixed by https://github.com/ajnisbet/opentopodata/commit/501296f38f1488f616b0f93405c3b4135559e084: after increasing the nginx buffer size the uwsgi buffer size becomes the bottleneck, which failes with a gateway error.

I increased both to the uwsgi max of 64k. That seems to be enough to get 3000 locations with the polyline format.

Any more locations that that would have to come as a POST request, which is a feature on my backlist.