drolbr / Overpass-API

A database engine to query the OpenStreetMap data.
http://overpass-api.de
GNU Affero General Public License v3.0
690 stars 90 forks source link

Invalid `latest_dir` `2024-05-20` points on directory which doesn't exists #728

Closed scorsi closed 1 month ago

scorsi commented 1 month ago

Hello, I don't know where to post for that, but the latest_dir in https://dev.overpass-api.de/clone/ points to 2024-05-20 instead of 2024-05-22 which causes failures on cloning because the directory doesn't exists.

scorsi commented 1 month ago

It got fixed yesterday.

mmd-osm commented 1 month ago

It got updated with the new date 2024-05-22 once the clone process has finished, but there's still some issue here.

drolbr commented 1 month ago

Thank you for reporting the problem. The underlying root cause is that the hard disk of dev.overpass-api.de got very slow (around 5-10 MB/s) over time and needs a replacement.

There have been considerations to move clone generation to a different machine which should have combined virtualization with Ansible, and various people did disagree how to do that in the right way. As a mitigation the clone generation has been reduced from daily to three times a week some time ago, and I will transition to twice a week now to give the slowing hard disk more time.

Another mitigation is to replicate as many files as possible client side. There is a partial implementation for the next release, but it has not been tested so far.

All in all, I'll leave this open as a task to ensure it can be tacked when one of the approaches to