False positives of Mapillary that reappear

[JorgeSanzS]

There is a problem with Mapillary detections. As it is normal there are false positives, the problem is that the marks as false positives and within days they appear again. For example this in the city of Salamanca. http://osmose.openstreetmap.fr/es/error/0e9703a2-79f1-186f-49cb-d3f6dd069b43 But it is the same with almost all those who are currently in Salamanca. This should be corrected because otherwise you cannot enter new information. It takes longer to discard what has already been discarded, than to put in what may be missing. Mapillary images lose their usefulness. Regards.

[frodrigo]

I already fix a issue on this few months ago. Do you think it happens on all, or just on some?

[JorgeSanzS]

In Salamanca I left it almost clean last week. All or almost all have reappeared. They were traffic light, surveillance camera, and bicycle parking. I think I removed something else but I'm not sure.

[JorgeSanzS]

Today they have reappeared. After 2 months marked as false positives. Almost all those seen in this area of the city of Salamanca. http://osmose.openstreetmap.fr/es/map/#zoom=15&lat=40.96899&lon=-5.65798&item=8360&level=1%2C2%2C3&tags=&fixable=

[frodrigo]

http://osmose.openstreetmap.fr/es/errors/?item=8360&class=2&country=spain_castilla_y_leon http://osmose.openstreetmap.fr/es/errors/false-positive?item=8360&class=2&country=spain_castilla_y_leon

[frodrigo]

Some marked as false positive may times (up to 8) : same location, same photo data but the marker hash changed.

Marker hash computation:

markers.uuid = ('{' || encode(substring(digest(%(source)s || '/' || %(class)s || '/' || %(subclass)s || '/' || %(elems_sig)s, 'sha256') from 1 for 16), 'hex') || '}')::uuid

• source, class not changing for sure.
  • subclass: can change on backend side.
  • elems_sig: empty on new object, probably not changing.

[frodrigo]

Fronend false positive

select lat, lon, count(*), array_agg(uuid), array_agg(subtitle->'en') from markers_status where item=8360 and class=2 and source_id=413298 group by lat, lon order by count(*);

40.9636215 | -5.6902626 | 8 | {eab5fc2f-0e92-6441-302f-87a9d0c8b0b3,7482d9e5-f2b9-7be0-8832-92b7f3e9ac09,1af47f19-441f-b170-6fc6-70a925d53335,a65589ca-cbdf-cb0d-e892-9b05d32a3669,767401d7-1616-0358-d920-80614a4d5117,6bda6730-0ac4-17e1-78be-f935ed060b61,0e8c7084-afd4-dab6-3946-66161fe5ae7c,55002443-4b26-5a66-8bed-2682cea969cb} | {"\"Observed on 2019-03-19\"","\"Observed on 2019-03-19\"","\"Observed on 2019-03-19\"","\"Observed on 2019-03-19\"","\"Observed on 2019-03-19\"","\"Observed on 2019-03-19\"","\"Observed on 2019-03-19\"","\"Observed on 2019-03-19\""}

Backend

select fields from ge_street_objects_2_leon0ab9_o where fields->'_y' like '40.9636%' and fields->'_x' like '-5.6902%';

Mapillary data download on 2020-01-17

"X"=>"-5.690285631748062", "Y"=>"40.96366224439598", "_x"=>"-5.690285631748062", "_y"=>"40.96366224439598", "value"=>"object--bike-rack", "accuracy"=>"5.4012337", "direction"=>"63.572735", "image_key"=>"9FFeS4oJRYMjvFmaQeg0rg", "last_seen_at"=>"2019-03-19T19:15:37.000Z", "first_seen_at"=>"2019-03-19T19:15:36.000Z"

The location is close bot not the same. Not sure if it is the same object.

40.9636215               | -5.6902626
40.96366224439598 | -5.690285631748062

Mapllary map does not show bike rack at this location https://www.mapillary.com/app/?lat=40.96371733432836&lng=-5.690144808955224&z=17&pKey=_Auj1q8jJqevRZs1pT589g&focus=map&detections=true

This photo should contain a object--bike-rack false positive, but not found it. https://www.mapillary.com/app/?lat=40.963755599972224&lng=-5.690051099972222&z=17&pKey=9FFeS4oJRYMjvFmaQeg0rg&focus=photo&detections=true&x=0.5572269336309565&y=0.4630327448123254&zoom=0

Still on Osmose http://osmose.openstreetmap.fr/es/map/#zoom=18&lat=40.963581&lon=-5.689604&item=8360&level=1%2C2%2C3&tags=&fixable=

Maybe values are moving on Mapillary side.

cc @cbeddow

[JorgeSanzS]

Appears in Mapillary as Point https://www.mapillary.com/app/?lat=40.96380116682806&lng=-5.690028061559133&z=17.152685299462483&pKey=_Auj1q8jJqevRZs1pT589g&focus=map&points=true&mapFeature%5B%5D=object--bike-rack

[frodrigo]

Right. I was looking for the wrong data layer. It may be not a bike parking, but it really look like one!

I have market it as false positive. Then we wait. Later I will look again as Mapillary data to see if there is some changes.

[JorgeSanzS]

Today many bugs from other classes have appeared again, removed in August http://osmose.openstreetmap.fr/es/map/#source=413298&item=8360&class=6&zoom=14&lat=40.97213&lon=-5.64605&level=1%2C2%2C3&tags=&fixable= http://osmose.openstreetmap.fr/es/map/#source=413298&item=8360&class=4&zoom=14&lat=40.97212&lon=-5.64603&level=1%2C2%2C3&tags=&fixable= http://osmose.openstreetmap.fr/es/map/#source=34016&item=8300&class=2&zoom=14&lat=40.96765&lon=-5.64586&level=1%2C2%2C3&tags=&fixable= These classes have fewer false positives. They can be more controllable.

[cbeddow]

New photos might cause problems with detecting the same false positive. Some things to consider also:

1) If making a hash based on the key of the detection/map feature key, or otherwise using the non-image keys, these may not be stable. If you look at using the image keys for making a hash it can be more stable, since these do not change. If same image keys were used to find the previous one and new one, then possible it's re-entering the system.

2) Mapillary verification API can accept a false positive report based on the detection key (for the object segmentation, but no the point object map feature key). See the "vote for an object detection" subsection: https://www.mapillary.com/developer/api-documentation/#object-detections (maybe this is already implemented?)

[frodrigo]

It is now back again, at a new location

40.9636324 -5.6902611

http://osmose.openstreetmap.fr/es/map/#zoom=18&lat=40.963581&lon=-5.689583&item=8360&level=1%2C2%2C3&tags=&fixable=

http://osmose.openstreetmap.fr/es/error/8e9917d9-158a-be3e-1204-8fcf473e56f3

Since location is not stable, I will try to compute a more stable hash, without using location.

[frodrigo]

Mark it as false positive again, with the new way to compute uniqueness.

[JorgeSanzS]

Thanks. I will mark the false positives

[JorgeSanzS]

the same thing happens again. They have reappeared

[Famlam]

I can confirm this, unfortunately, for multiple regions in NL