drolbr / Overpass-API

A database engine to query the OpenStreetMap data.
http://overpass-api.de
GNU Affero General Public License v3.0
716 stars 90 forks source link

Long deleted object in lz4 but not other instances #563

Closed hdrs closed 4 years ago

hdrs commented 4 years ago

I asked the question here first and somebody suggest I ask it in this forum.

An act of vandalism created "Creedon Republic", a large country object, shown here way/617866825.

It was removed on 2018-08-19 with this changeset changeset/61786055.

This object is still present in lz4.overpass-api.de, but not in other overpass servers.

This command gets the object for me:

wget -O - "http://lz4.overpass-api.de/api/interpreter?data=[out:json];is_in(38.49464,-75.28586);out body;" | more

It looks like this in the results:

... { "type": "area", "id": 3017866825, "tags": { "admin_level": "1", "boundary": "administrative", "name": "Creedon Republic", "place": "country" } },...

This one does not:

wget -O - "http://z.overpass-api.de/api/interpreter?data=[out:json];is_in(38.49464,-75.28586);out body;" | more

Is there some misunderstanding on my part as to what the instances are supposed to contain? Or is this a problem with lz4?

Thanks in advance.

mmd-osm commented 4 years ago

That's pretty much a duplicate of #285 as already mentioned by someone else on that help site.

As you've probably noticed, that issue was marked as won't fix. What was stated in https://github.com/drolbr/Overpass-API/issues/285#issuecomment-234764265 still applies today.

hdrs commented 4 years ago

That's pretty much a duplicate of #285 as already mentioned by someone else on that help site.

As you've probably noticed, that issue was marked as won't fix

So would your recommended policy be just to never use lz4 going forwards? I can certainly just do that as unilateral policy but if there is this cruft in other instances I am stuck.

My dominant use is on our private ovp instances, which are made from more recent planet files and presumably do not have the issue as much. But for some tasks I have to use public ones.

mmd-osm commented 4 years ago

I would recommend to use pivotto determine the respecitve way/relation to filter out no longer existing areas. This will work irrespective of the instance you're using.

Example:

[out:json];
is_in(38.49464,-75.28586);
wr(pivot);
map_to_area;
out;
hdrs commented 4 years ago

OK, thanks.

mmd-osm commented 4 years ago

@hdrs : can you close this issue in case there are no further questions? Thanks

hdrs commented 4 years ago

@mmd-osm : Yes, I will close now.

We respect that the decision was taken not to fix by the system owners but we regard this behavior as rendering the system(s) unfit for usage. So this is a real and an extant problem for us.

Thanks for your help in the usage patch but we regard that as also unfit for routine usage.

mmd-osm commented 4 years ago

Well, this is open source, you can set up your own instance and re-create areas from scratch from time to time.

hdrs commented 4 years ago

We have 5 instances running now at full load for our core internal task, but as I mentioned, for some (smaller and very particular) tasks we cannot hit our internal ones.

We have (I think) run global instances before, but not more than one at a time.

Right now we are running 5 subset instances.

Area subsetting the planet file yields a weird edge case bug that makes the instance not fit for purpose on some tasks. Worse in fact than lz4 is right now. If you are a good person to ask about that I can try to set up a publishable circumstance to reproduce it. Or describe it in wishy washy terms.

mmd-osm commented 4 years ago

You mean you run some instances on some planet extract and try to create areas for it? Where did you got your extracts from? Do you run some updates on them? What's the kind of issue you see with areas in that case?

hdrs commented 4 years ago

Should I start a new issue and document it suitably?

mmd-osm commented 4 years ago

Yes, please do so, if you can reproduce it. Thanks.

hdrs commented 4 years ago

I cannot easily reproduce it in a public setting. I will do some tests public instances that use only a subset and see if I can get it to happen.

I can probably also try to write it up with pictures and objects.

mmd-osm commented 4 years ago

You can also ignore what the other user wrote about a „cleaning policy“. There’s no such thing, the only way to get rid of stale areas is to delete all area files on the server and start from scratch. Nobody I know of is doing anything like that.