drolbr / Overpass-API

A database engine to query the OpenStreetMap data.
http://overpass-api.de
GNU Affero General Public License v3.0
692 stars 90 forks source link

Area - repeated/inconsistent tag key/value pairs #672

Closed mmd-osm closed 1 year ago

mmd-osm commented 1 year ago

This looks like a regression in 0.7.58: in some cases, the same key-value pair is repeated multiple times for the same area:

area["ISO3166-1"="IT"];
out;

Sometimes, there are even deviating values for the same key, e.g. "operator" key:

area["ISO3166-1"="AD"];
area[amenity="fuel"](area);
area._(id:3602660789);
out center meta;
  <area id="3602660789">
    <tag k="amenity" v="fuel"/>
    <tag k="amenity" v="fuel"/>
    <tag k="brand" v="BP"/>
    <tag k="brand" v="BP"/>
    <tag k="brand:wikidata" v="Q152057"/>
    <tag k="brand:wikidata" v="Q152057"/>
    <tag k="brand:wikipedia" v="en:BP"/>
    <tag k="brand:wikipedia" v="en:BP"/>
    <tag k="fuel:diesel" v="yes"/>
    <tag k="fuel:diesel" v="yes"/>
    <tag k="fuel:octane_100" v="yes"/>
    <tag k="fuel:octane_100" v="yes"/>
    <tag k="fuel:octane_95" v="yes"/>
    <tag k="fuel:octane_95" v="yes"/>
    <tag k="name" v="BP"/>
    <tag k="name" v="BP"/>
    <tag k="operator" v="BP"/>
    <tag k="operator" v="ΚΑΛΥΨΩ ΜΟΝ. Ε.Π.Ε. Νο 72"/>
    <tag k="payment:hellenic_fuels_SA" v="yes"/>
    <tag k="payment:hellenic_fuels_SA" v="yes"/>
    <tag k="type" v="multipolygon"/>
    <tag k="type" v="multipolygon"/>
  </area>

By the way, I would consider area...(area) undefined / unspecified behavior, and would propose the following alternative:

area["ISO3166-1"="AD"];
wr[amenity="fuel"](area);
out center meta;
drolbr commented 1 year ago

Recreating area 3602660789 on the public instances has fixed the issue for that area.

Have you any particular reason why this should affect only version 0.7.58, or do we simply never have observed it before? Is there a setup to reproduce the issue on area creation?

mmd-osm commented 1 year ago

That's a good question. I can only offer a few data points for a number of public instances:

Test query:

area["ISO3166-1"="IT"];
out;

It would be interesting to know, if areas have been created from scratch on both lz4 and z, or if they have been copied from one instance to the other.

drolbr commented 1 year ago

Fixed in b0c4acbb1aca720244f95d62ad14fc85fb12467e

Some dead code lured me yesterday event into to put the code there instead of the proper place. The fix is simple: the search criterion for finding the outdated tags has been unsorted, resulting in missing some tags to delete during readout from the database. The algorithm assumed that there were no tags to delete and added the new tags along the old ones. It is a mandelbug nonetheless because the unwanted data must have been inserted in a previous run, and independent objects with a suitably conflicting index must be present to confuse the index order.

mmd-osm commented 1 year ago

By the way, I think there's another prepare_tags function in tags_updater.h, that looks like another incarnation of the same dead code.