Using results from osm-compare as attributes

mapbox / gabbar

Guarding OpenStreetMap from harmful edits using machine learning

MIT License

19 stars 7 forks source link

Using results from osm-compare as attributes #38

Closed bkowshik closed 7 years ago

bkowshik commented 7 years ago

We can potentially use the comparators in osm-compare as attributes in Gabbar. The workflow would look like:

Given a real changeset with both new version and old version of features
Pass every feature to osm-compare and collect back results
Ex: Changeset has 2 features where name of feature does not match name on Wikidata

cc: @amishas157 @batpad

bkowshik commented 7 years ago

Sync comparators

added_place
common_tag_values
draggedHighwayWaterway
large_building
major_name_modification
major_road_changed
null_island
pokemon_edits

Async comparators (uses callback)

invalid_name
invalid_tag_combination
invalid_tag_modification

External API requests

name_unmatches_with_wikidata
feature_overlap
water_feature_by_new_mapper
pokemon_footway

bkowshik commented 7 years ago

Following are some compatibility issues:

Real changeset features have all osm attributes in feature.properties.tags. But, compare functions expect all attributes in feature.properties
Some flags like deleted do not exist in the real changeset stream. Ex: added_website.js

bkowshik commented 7 years ago

Following is a summary of compare functions results on the validation dataset.

Total changesets: 2,582
Total features: 257,000

Compare functions	Feature flagged
cf_common_tag_values	44
cf_added_place	27
cf_pokemon_edits	2
cf_null_island	0
cf_dragged_highway_waterway	0
cf_large_building	0
cf_major_name_modification	0
cf_major_road_changed	0

Feel like number of features flagged by compare functions are too few.
Models give feature importance scores and comparators had extremely low scores. Which in-turn means they are not playing a significant role in improving the quality of predictions.

batpad commented 7 years ago

@bkowshik this seems to me that we should revisit the idea of more "tagger" compare functions which serve the function of more general tagging of the types of changes in a changeset, and see if that helps at all to increase the relevance? Happy to discuss.

bkowshik commented 7 years ago

Closing for now. Let's revisit this in a couple of weeks time.