search

wiktorn / Overpass-API

Overpass API docker image
MIT License
134 stars 48 forks source link

404 Not Found errors for retrieving Overpass data #62

Closed kmehta-nuro closed 3 years ago

kmehta-nuro commented 3 years ago

I am running the clone example from the README. The docker image starts successfully, but after downloading data, I have many 404 Not Found errors for retrieving from the Overpass server.

kmehta@kmehta-desktop:~$ sudo docker run   -e OVERPASS_META=yes   -e OVERPASS_MODE=clone   -e OVERPASS_DIFF_URL=https://planet.openstreetmap.org/replication/minute/   -v /big/docker/overpass_clone_db/:/db   -p 12346:80   -i -t   --name overpass_world   wiktorn/overpass-api
No database directory. Initializing
--2021-01-19 18:46:58--  http://dev.overpass-api.de/api_drolbr//trigger_clone
Resolving dev.overpass-api.de (dev.overpass-api.de)... 95.217.37.171, 2a01:4f9:2b:2fd4::2
Connecting to dev.overpass-api.de (dev.overpass-api.de)|95.217.37.171|:80... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

    The file is already fully retrieved; nothing to do.

--2021-01-19 18:47:01--  https://dev.overpass-api.de/clone//2021-01-15/replicate_id
Resolving dev.overpass-api.de (dev.overpass-api.de)... 95.217.37.171, 2a01:4f9:2b:2fd4::2
Connecting to dev.overpass-api.de (dev.overpass-api.de)|95.217.37.171|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-01-19 18:47:02 ERROR 404: Not Found.

Fetching nodes.bin
--2021-01-19 18:47:02--  https://dev.overpass-api.de/clone//2021-01-15/nodes.bin
Resolving dev.overpass-api.de (dev.overpass-api.de)... 95.217.37.171, 2a01:4f9:2b:2fd4::2
Connecting to dev.overpass-api.de (dev.overpass-api.de)|95.217.37.171|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-01-19 18:47:02 ERROR 404: Not Found.
wiktorn commented 3 years ago

It might be the case that you didn't manage to download all the files of the clone, before it was removed from server, as it looks like you're trying download a bit old files. Anyway - for me the first request trigger_clone returns the date 2020-01-19 and from the logs it looks like the file already exists, so you try to download to directory that was already partly download. The script tries to continue, but it will continue as long as the files are available on the server, which is no longer the case.

You need to start over with empty directory.

kmehta-nuro commented 3 years ago

Ah got it. Do the files get updated every day? The initial download takes me ~20 hours

wiktorn commented 3 years ago

Yes, files are updated every day.

kmehta-nuro commented 3 years ago

Ok, so I need the entire initial download to finish within one day, but after that the container can continue running?

wiktorn commented 3 years ago

Exactly

mitkodev commented 1 year ago

hey @wiktorn, sorry to open up an old issue but at the moment http://dev.overpass-api.de/api_drolbr//trigger_clone returns https://dev.overpass-api.de/clone//2023-05-08, but https://dev.overpass-api.de/clone//2023-05-08/replicate_id is currently missing as it returns 404. How do I resolve this?

wiktorn commented 1 year ago

@mitkodev I guess you were just unlucky. Now trigger_clone returns 2023-05-09 and the files are there.