False Positive: Small highway group apart from the main network or with insufficient access upstream

AnkEric commented 4 years ago

My - first - impression:

New issue today 2020-03-16
Many, too many, positives. All of the the Netherlands is covered with these warnings.
Might be IF a highway ALSO connects to another class of highway (f.i. tertiary connects to track, even on junction where tertiary does continue as tertiary)

http://osmose.openstreetmap.fr/en/map/?#zoom=16&lat=51.9729&lon=5.00676&item=1210&level=1%2C2%2C3&tags=&fixable=

source = 32963 item = 1210 class = 4 lat lon = 51.9741460 5.0019882

/ AnkEric

frodrigo commented 4 years ago

I agree something wrong, but it is not new

http://osmose.openstreetmap.fr/en/errors/?country=netherlands*&item=1210 imagen

AnkEric commented 4 years ago

Oké, thanks. "Not new" --> More correct: "I just noticed today for the first time". Must have had my eyes closed. And pressed for time; -) Sloppy ticket, sorry!

frodrigo commented 4 years ago

Missing ways in extracts Eg. netherlands_utrecht have 7105527, but not 509592657

cc @jocelynj @Marc-marc-marc

jocelynj commented 4 years ago

I've found that Provinces in NL were changed in january 2019, and polygons used on download.openstreetmap.fr weren't updated. So, we lost alignement between generated extracts and boundaries used by osmose.

I will regenerate extracts on download.openstreetmap.fr.

jocelynj commented 4 years ago

I've looked at a few errors in NL, and they seems to be real issues.

For example: http://osmose.openstreetmap.fr/en/map/?#zoom=18&lat=51.914685&lon=4.459046&item=1210&level=1%2C2%2C3&tags=&fixable= It is not connected to the nearby road "Claes de Vrieselaan"

AnkEric commented 4 years ago

This example is OK (for Osmose), it's Error on OSM:

( railway=tram ) > ( node 44374198 ) > ( highway=residential ) > ( highway=residential )

( railway=tram ) > ( node 44374198 ) > ( highway=residential ) should not be connected. So first segment Robert Fruinstraat (between railway and residential) is inaccessible.

I will fix tomorrow (in case you want to review...)

For now:

fixme = Disconnect Robert Fruinstraat from (railway=tram)

frodrigo commented 4 years ago

http://osmose.openstreetmap.fr/en/errors/?country=netherlands_*&item=1210&level=1%2C2%2C3 http://osmose.openstreetmap.fr/en/errors/graph.png?country=netherlands_*&item=1210&level=1%2C2%2C3

Update sill in progress, wait and see.

AnkEric commented 4 years ago

Ok @frodrigo, @jocelynj I will wait...

But also, meanwhile...

I don't understand.

What did we learn from: "Why: 'Unconnected cycleway'?" https://github.com/osm-fr/osmose-backend/issues/337 It took 2 months before this issue was resolved. Now we are one month behind actual status. Waiting for an update lasted one extra month in 2018.

Is it possible to - temporarily - disable False Positive tests? Now it takes (too much) time before I understand I should not try to resolve these False Positives. Each "not-connected highway/cycleway" might be a showstopper for navigation during a vacation trip.

I know "Provinces in NL were changed in January 2019" (my home province "Utrecht"), but why is this causing the issue? This happened one year ago.

Does ("OSM tiles servers are overloaded") have anything to do with this issue? NL extracts are overdue by ± 10 hours each day. If I am lucky.

Looking at the graph "not-connected highway/cycleway netherlands" it is suggested the issue goes back to end 2018? Is it possible to do an "Automated Osmose Validation Test" on extract status?

Marc-marc-marc commented 4 years ago

I learn from #337 and this one, that :

changes in administrative boundaries can corrupt extracts (because if you request a municipality from the administrative entity to which it no longer belongs, it cannot produce a good result).
extracts can be corrupted for another still unknown one, we are looking for a way to detect this case automatically since, as you can see, it can take a year for a human to detect it. if you have an idea of how to do it, don't hesitate

Marc-marc-marc commented 4 years ago

osmium detect it time /usr/bin/osmium check-refs utrecht.osm || echo ko

[======================================================================] 100% There are 7087591 nodes, 1028639 ways, and 7864 relations in this file. Nodes in ways missing: 2695

real 0m38.320s user 0m38.093s sys 0m1.200s ko

frodrigo commented 4 years ago

We resolve issues in Utrecht, but only here.

@jocelynj do you regenerate the whole NL, or not?

jocelynj commented 4 years ago

I've regenerated all provinces of NL.

jocelynj commented 4 years ago

osmium detect it time /usr/bin/osmium check-refs utrecht.osm || echo ko

There are 7087591 nodes, 1028639 ways, and 7864 relations in this file. Nodes in ways missing: 2695 ko

I don't see how this is related to the issue here.

Yes, there are missing nodes in the extracts, but:

they should be all outside of the administrative polygon + a buffer.
they shouldn't impact osmose analysers, as osmose only keep errors from inside the administrative polygon.

Marc-marc-marc commented 4 years ago

I thought extracts were complete ordinarily. if they are not, the most practical criteria to automate the detection of a "more incomplete than normal" extract is to detect the explosion of the number of anomalies in this analysis.

frodrigo commented 4 years ago

I look most at the remaining issues, and the others are legitimate.

The there lot of detailed data, and then lot of issues.

@AnkEric I do not understand what you mean about false-positive.

Data are analysed by small parts (extracts), and how part are spliced may raise issues, especially when changing the split.

There is no link with tile server.

Netherlands check start on updated extract (max lag of 1min). The check is run every 24h.

If you have other issues, please open a new issue.

osm-fr / osmose-backend

False Positive: Small highway group apart from the main network or with insufficient access upstream #801