search

wiktorn / Overpass-API

Overpass API docker image
MIT License
134 stars 48 forks source link

Ignore corruption in changes.osm #56

Closed CJxD closed 3 years ago

CJxD commented 3 years ago

If changes.osm is corrupt, osmium fileinfo will return exit code 1 and the script will terminate, but the diff file will still exist. As a result, the changes will never be applied. This small change simply returns "" if the timestamp can't be read, and the file will be skipped.

wiktorn commented 3 years ago

Thanks for PR.

I have some doubts about this change, as the check is after we have downloaded the files from server. If we remove them, it may mean that we will loose some changes by accident.

Maybe the approach would be to copy the /db/replicate_id file, and if we do such change, revert the file to the state before the script has run. This way, we will try to download the same changeset once again.

What was the real-world use case that you've encountered? What was in the changes.osm file and how did it happen?

CJxD commented 3 years ago

Good point, yes there should be a rollback of /db/replicate_id in the event on an unsuccessful download.

So the reason I did this is because today I found out that my server had not been updated for several months because a changes.osm file was part-downloaded. In particular, it was missing at least the last </osm> tag, which sent it into a retry cycle.