search

ProjectSidewalk / sidewalk-quality-analysis

An analysis of Project Sidewalk user quality based on interaction logs
5 stars 3 forks source link

Label Correctness Visualization #23

Open daotyl000 opened 5 years ago

daotyl000 commented 5 years ago

Could make a google maps view of all verified labels with green being correct and red being incorrect. This would allow us to see if the region/location of Seattle has an effect on the level of correct labels being placed.

jonfroehlich commented 5 years ago

Again, this captures a suggestion I made to @daotyl000. Would be nice to have a quick top-down map visualization of correct vs. incorrect labels to see if we can spot some geographic patterns.

daotyl000 commented 5 years ago

Two of the users ( 57% & 63% accuracy) saw most of their incorrect labels in the industrial district while their labels in other neighborhoods were fairly accurate. The third user (38% accuracy) saw about a 50% correctness in South Park (industrial area in South Seattle) but a very how correctness percentage in Wedgewood (residential neighborhood in North Seattle

57% accuracy Screen Shot 2019-07-10 at 11 27 50 AM 38% accuracy: Screen Shot 2019-07-10 at 11 27 59 AM 63% accuracy Screen Shot 2019-07-10 at 11 28 11 AM

daotyl000 commented 5 years ago

Here are zooms of the neighborhoods with the highest incorrect rates Screen Shot 2019-07-10 at 11 48 26 AM Screen Shot 2019-07-10 at 11 49 28 AM Screen Shot 2019-07-10 at 11 50 46 AM Screen Shot 2019-07-10 at 11 51 06 AM

daotyl000 commented 5 years ago

Updated with the new bad users(10) wide shorts & close up on all regions with high incorrect rates

Screen Shot 2019-07-17 at 3 22 02 PM Screen Shot 2019-07-17 at 3 22 21 PM Screen Shot 2019-07-17 at 3 23 30 PM Screen Shot 2019-07-17 at 3 24 05 PM Screen Shot 2019-07-17 at 3 24 16 PM Screen Shot 2019-07-17 at 3 24 25 PM Screen Shot 2019-07-17 at 3 26 12 PM Screen Shot 2019-07-17 at 3 26 40 PM Screen Shot 2019-07-17 at 3 26 45 PM

jonfroehlich commented 5 years ago

One option here would be to create an admin vis that shows correct and incorrect labels (in green and red, respectively).

If we wanted to support per label type, we could potentially use the label type's outline color and then the fill be red/green (for correctness) or something...

We could also have a tool that visualizes labels per user...