/api/interpreter does not respond after initial clone, times out with 504

ctoth commented 2 years ago

With the following docker-compose.yml file, after initially cloning and then restarting the service, the logs indicate that everything starts up correctly. When I make an API call however, the service only responds with an HTTP 504 gateway timeout.

volumes:
  overpass-db:

version: "3.8"
services:
  overpass_api:
    image: wiktorn/overpass-api
    restart: always
    ports:
      - "8080:80"
    volumes:
      - overpass-db:/db
    environment:
      - OVERPASS_RULES_LOAD="50"
      - OVERPASS_FASTCGI_PROCESSES="8"

The logs after the second startup of the service (after clone)

Starting overpass-server_overpass_api_1 ... done                                                                        
Attaching to overpass-server_overpass_api_1                                                                             
overpass_api_1  | Starting supervisord process                                                                          
overpass_api_1  | 2022-01-10 22:01:26,383 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.                                                                                                              
overpass_api_1  | 2022-01-10 22:01:26,384 INFO supervisord started with pid 1                                           
overpass_api_1  | 2022-01-10 22:01:27,387 INFO spawned: 'overpass_dispatch' with pid 11                                 
overpass_api_1  | 2022-01-10 22:01:27,389 INFO spawned: 'nginx' with pid 12                                             
overpass_api_1  | 2022-01-10 22:01:27,392 INFO spawned: 'fcgiwrap' with pid 13                                          
overpass_api_1  | 2022-01-10 22:01:27,394 INFO spawned: 'update_overpass' with pid 14                                   
overpass_api_1  | 2022-01-10 22:01:27,397 INFO spawned: 'dispatcher_areas' with pid 15                                  
overpass_api_1  | 2022-01-10 22:01:27,398 INFO spawned: 'areas_rules' with pid 16                                       
overpass_api_1  | /app/bin /                                                                                            
overpass_api_1  | 2022-01-10 22:01:28,405 INFO success: overpass_dispatch entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)                                                                                         
overpass_api_1  | 2022-01-10 22:01:28,405 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)                                                                                                     
overpass_api_1  | 2022-01-10 22:01:28,406 INFO success: fcgiwrap entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)                                                                                                  
overpass_api_1  | 2022-01-10 22:01:28,406 INFO success: update_overpass entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)                                                                                           
overpass_api_1  | 2022-01-10 22:01:28,406 INFO success: dispatcher_areas entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)                                                                                          
overpass_api_1  | 2022-01-10 22:01:28,406 INFO success: areas_rules entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)                                                                                               
overpass_api_1  | <?xml version="1.0" encoding="UTF-8"?>                                                                
overpass_api_1  | <osm version="0.6" generator="Overpass API 0.7.57 93a4d346">                                          
overpass_api_1  | <note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>                                                                                                         
overpass_api_1  | <meta osm_base=""/>                                                                                   
overpass_api_1  |                                                                                                       
overpass_api_1  | After 0h0m15s: in "query", part 4, on line 13. Stack: 0 of 0 0 of 0

wiktorn commented 2 years ago

Does it work if you instead of cloning, download a small subset (such as Monaco)?

I don't even see the 504 response in the logs here so I'm not sure, if the requests reaches this container. Similar like this 404 that I've simulated below:

172.17.0.1 - - [10/Jan/2022:22:11:05 +0000] "GET /nonexistent HTTP/1.1" 404 153 "-" "curl/7.79.1"

ctoth commented 2 years ago

I will try a subset next. The timeouts were coming from the health check it looked like. If I open a browser to a nonexistent URL I get the following in the logs:

overpass_api_1  | 2022/01/10 22:18:10 [error] 33#33: *35 open() "/usr/share/nginx/html/nonexistent" failed (2: No such file or directory), client: 172.18.0.1, server: localhost, request: "GET /nonexistent HTTP/1.1", host: "osm.audiom.xrnavigation.io"                                                                                                              
overpass_api_1  | 172.18.0.1 - - [10/Jan/2022:22:18:10 +0000] "GET /nonexistent HTTP/1.1" 404 188 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"                     
overpass_api_1  | 2022/01/10 22:18:10 [error] 33#33: *35 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 172.18.0.1, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "osm.audiom.xrnavigation.io", referrer: "https://osm.audiom.xrnavigation.io/nonexistent"                                                  
overpass_api_1  | 172.18.0.1 - - [10/Jan/2022:22:18:10 +0000] "GET /favicon.ico HTTP/1.1" 404 188 "https://osm.audiom.xrnavigation.io/nonexistent" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"

The failed health checks look like:

overpass_api_1  | 2022/01/10 22:22:36 [error] 33#33: *9 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 127.0.0.1, server: localhost, request: "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1", upstream: "fastcgi://unix:/nginx/fcgiwrap.socket", host: "localhost"                                   
overpass_api_1  | 127.0.0.1 - - [10/Jan/2022:22:22:36 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 504 494 "-" "curl/7.64.0"

wiktorn commented 2 years ago

Any results with smaller subset?

The error message suggests problem with connecting to dispatcher, but there are no errors in the log showing, that it indeed failed. That's strange.

MartinezCatriel commented 1 year ago

Hi, I had a similar issue and fixed it by initializing a VM in Docker with more ram. I was using a VM with 4GB and when I set 8GB it worked perfectly (considering that I mounted Argentina region). Did you consider increasing the RAM?

mmisztal1980 commented 1 year ago

SImilar results over here, running on a 16GB VM in Docker

version: '3.5'
services:
  overpass:
    container_name: overpass
    image: wiktorn/overpass-api:0.7.56.9
    ports:
      - 6080:80
    volumes:
      - overpass-db:/db
    environment:
      - OVERPASS_META=no
      - OVERPASS_MODE=init
      - OVERPASS_PLANET_URL=http://download.geofabrik.de/europe/monaco-latest.osm.bz2
      - OVERPASS_DIFF_URL=http://download.openstreetmap.fr/replication/europe/monaco/minute/
      - OVERPASS_UPDATE_SLEEP=3600
      - OVERPASS_USE_AREAS=false
    healthcheck:
      test: ["CMD-SHELL", "curl -qf 'http://localhost/api/interpreter?data=[out:json];node(1);out;' | jq '.generator' | grep -q Overpass || exit 1"]
      start_period: 48h
volumes:
    overpass-db:

and all I'm getting is:

overpass  | 172.20.0.1 - - [02/Mar/2023:16:08:15 +0000] "POST /api/interpreter HTTP/1.1" 504 635 "-" "-"
overpass  | 172.20.0.1 - - [02/Mar/2023:16:08:18 +0000] "POST /api/interpreter HTTP/1.1" 504 635 "-" "-"
overpass  | 172.20.0.1 - - [02/Mar/2023:16:08:22 +0000] "POST /api/interpreter HTTP/1.1" 504 635 "-" "-"

@wiktorn any hints?

wiktorn commented 1 year ago

@mmisztal1980 I tried your docker-compose and started it using:

$ docker-compose up

Once it completed to initialize the database, I started it again and

curl -v "http://localhost:6080/api/interpreter?data=\[out:json\];node(1);out;"
*   Trying 127.0.0.1:6080...
* Connected to localhost (127.0.0.1) port 6080 (#0)
> GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1
> Host: localhost:6080
> User-Agent: curl/7.82.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.21.6
< Date: Fri, 03 Mar 2023 21:53:36 GMT
< Content-Type: application/json
< Transfer-Encoding: chunked
< Connection: keep-alive
<
{
  "version": 0.6,
  "generator": "Overpass API 0.7.56.9 76e5016d",
  "osm3s": {
    "timestamp_osm_base": "2023-03-03T07:04:24Z",
    "copyright": "The data included in this document is from www.openstreetmap.org. The data is made available under ODbL."
  },
  "elements": [

  ]
}

So works perfectly fine. On console also looks good:

overpass  | 172.18.0.1 - - [03/Mar/2023:21:53:36 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 298 "-" "curl/7.82.0"

What kind of the POST requests you're sending to your instance?

wiktorn / Overpass-API

/api/interpreter does not respond after initial clone, times out with 504 #82