Update: Deleted nodes don't get removed

[sebisteiner]

Hi

I'm running this image with the Switzerland data files from Geofabrik:

• https://download.geofabrik.de/europe/switzerland.html
• https://download.geofabrik.de/europe/switzerland-updates/

Yesterday I made some changes in OSM.

• Deleted a node
• Changed tags of another node

See here: https://www.openstreetmap.org/changeset/109912835

The changes are now visible in my data set. But the deleted node did not get removed. On the public Overpass API Server (overpass-api.de/api) it is gone.

Any idea why this is not working?

The deleted node is included in this diff file: https://download.geofabrik.de/europe/switzerland-updates/000/003/065.osc.gz

  <delete>
    <node id="4966471722" version="3" timestamp="2017-12-15T06:53:31Z" lat="47.0737695" lon="7.3052828">
      <tag k="name" v="Billard - Bar Lyss"/>
      <tag k="email" v="info@billard-bar-lyss.ch"/>
      <tag k="amenity" v="bar"/>
      <tag k="name:de" v="Billard-Bar Lyss"/>
      <tag k="name:en" v="Billard-Bar Lyss"/>
      <tag k="name:fr" v="Billard-Bar Lyss"/>
      <tag k="website" v="http://www.billard-bar-lyss.ch"/>
      <tag k="addr:street" v="Juraweg"/>
      <tag k="addr:postcode" v="3250"/>
      <tag k="opening_hours" v="Tu-Th 17:30-23:30; Fr-Sa 17:30-00:30; Su 15:00-23:30"/>
      <tag k="addr:housenumber" v="1a"/>
    </node>
  </delete>

Sebastian

[wiktorn]

No idea. Which version of the image you're running?

[sebisteiner]

Hi

I'm running the latest version: (sha256:c4da[...])

I think it's the same problem as described here: https://github.com/wiktorn/Overpass-API/issues/39#issuecomment-625755849

[sebisteiner]

I have no idea how Overpass works, so maybe I'm talking rubbish now... 😄

But here someone says that update_database should only be used for the initial setup. Subsequent changes should be done with update_from_dir.

https://gis.stackexchange.com/questions/339758/updating-overpass-api-database-with-local-changes-file/339768#339768

Could this be an issue?

https://github.com/wiktorn/Overpass-API/blob/master/bin/update_overpass.sh#L63

[wiktorn]

Hmm.. might be. I'll adapt to that today. Thank you for pointing out.

[wiktorn]

I've made the necessary changes, the docker images are updated. Though as I've gone through Overpass code, I'd assume, that osmium has already sorted the contents in required way. So please recreate your image, update database (or if possible, try to reapply the same change rewinding replication_id).

Let me know, if you'll face the same issue again

[sebisteiner]

Unfortunately it still does not work...

I updated my image and recreated the DB from scratch.

Afterwards I deleted an obsolete node in OSM. The node (2194992680) is included in this changeset: https://download.geofabrik.de/europe/switzerland-updates/000/003/086.osc.gz

The changeset was successfully applied (checked replicate_id) but the node is still there.

Any idea how to debug/analyze this?

[wiktorn]

It would be great, if you could share:

• original dump used to created overpass db
• all environmental variables

Also, if you could paste the response of: curl "http://localhost:${PORT}/api/interpreter?data=\[out:json\];node(2194992680);out;"

Could be helpful. But having original dump + changesets + node that doesn't get removed, I guess that's something, that really can help narrow this down.

[sebisteiner]

That's how I start my container.

$ docker run \
> -e OVERPASS_MODE=init \
> -e OVERPASS_PLANET_URL=https://download.geofabrik.de/europe/switzerland-latest.osm.bz2 \
> -e OVERPASS_DIFF_URL=https://download.geofabrik.de/europe/switzerland-updates/ \
> -e OVERPASS_FLUSH_SIZE=1 \
> -v /var/data/overpass/switzerland/:/db:Z \
> -p 80:80 \
> -d \
> --name overpass-switzerland wiktorn/overpass-api

And that's the response of the overpass query:

{
  "version": 0.6,
  "generator": "Overpass API 0.7.56.9 76e5016d",
  "osm3s": {
    "timestamp_osm_base": "2021-09-09T20:16:38Z",
    "copyright": "The data included in this document is from www.openstreetmap.org. The data is made available under ODbL."
  },
  "elements": [
    {
      "type": "node",
      "id": 2194992680,
      "lat": 47.1006992,
      "lon": 7.3271873,
      "tags": {
        "addr:housenumber": "46",
        "addr:postcode": "3292",
        "addr:street": "Bahnhofstrasse",
        "amenity": "restaurant",
        "name": "Rössli",
        "website": "http://www.roessli-busswil.ch/"
      }
    }
  ]
}

original dump used to created overpass db

It's the planet file linked above. But I don't have yesterdays version (where the node is still contained) stored on my disk anymore.

[wiktorn]

Thank you. Given this input I was able to trace it down to improperly named diffs generated by pyosmium-get-changes. Fix commited and images corrected.

Thank you for helping tacking this down.