mapbox / gabbar

Guarding OpenStreetMap from harmful edits using machine learning
MIT License
19 stars 7 forks source link

Increasing number of changesets flagged #4

Closed bkowshik closed 7 years ago

bkowshik commented 7 years ago

For the last two days that gabbar has been live on osmcha-staging, it has flagged less than 30 changesets as problematic everyday. In the real-world, there could potentially be more changesets that are harmful. So:


cc: @rodowi

danstowell commented 7 years ago

Hi - an osmcha user here. I see many dozens of "flagged by gabbar", and I'm only looking at a localish area - I don't think you want to increase the amount too much.

You say "bring accuracy down a little", but that can't be right. You want to increase the accuracy, in general. I guess you mean you want to change the threshold, i.e. change the balance of false-positive and false-negative. But I go back to my first point - do consider the risk of flooding osmcha!

bkowshik commented 7 years ago

:wave: @danstowell happy to meet a fellow osmcha user. 😃

gabbar is a super-experimental machine learning based changeset classifier to identify problematic edits on OpenStreetMap. We deployed to osmcha-django-production over the weekend, but it turned out to be super noisy and as a result we have brought it down. We will take the next couple of days to iterate on the classifier and bring back an improved version of gabbar to osmcha-django.

I guess you mean you want to change the threshold.

Thank you for clarifying. I meant changing the threshold of the classifier.

bkowshik commented 7 years ago

No next actions here. Closing.