Beta test on Camiguin island to gather feedback on prioritization and validation

[maning]

This tasking manager project is for evaluating the mapping workflow of tasks-assisted.hotosm.org + RapiD.

• Use FB HRSL (with a buffer of 200m) during tasking preparation to identify areas to map.
• Use RapiD to improve the road coverage by:
  • validate roads detected by the tool;
  • add missing roads not detected by RapiD
  • align and improve shapes of existing roads;
  • tag roads accordingly and;
  • fix validation issues ( i.e. overlapping geometries, too many nodes, etc.).

Mapping roads with RapiD in Camiguin Island: https://tasks-assisted.hotosm.org/project/9

After the testing of workflow we would like to answer the following:

• Is it effective to use the HRSL data to prioritize areas to map?
• Does RapiD increased the perceived efficiency of mapping according to the users?
• What are common mapping mistakes we found during validation and how can we avoid them.

@govvin Please lead the first stage of mapping with your CWTS++ students. @adisadit Thank you for preparing the TM project!

[govvin]

Impressions and observations from CWTS++ volunteers for the Camiguin test task

Christine F:

for me po yung RapiD ay malaking tulong sa OSM dahil po natutukoy nya po agad yung road and ang need mo lang gawin is double check kung tama po ba ang pag lapat ng AI or RapiD app sa roads na tinukoy ni RapiD.

Marlon R:

As what i observed when i started to map, it is now very simply to identify a road rather than what we used to before because RapiD can automatically detect a road which symbolize as magenta line so that i can easily set it as a road, connect it to another road or just delete it if it is unnecessary. It was fun to use RapiD as editor and map with AI. I can just finish the assign task much earlier than before.

Leo M:

Images are a bit clearer compared to [Nepal] mapping. Roads are easier to track thanks to RapiD. Encountered areas that has nothing to be mapped. Encountered a magenta line that is out of the work area boundary. Did not encounter magenta lines that is within the work area boundary. Roads are easier to trace compared to the Nepal tasks. Today's tasks are easily completed compared to the Nepal tasks last meeting. Also there are few tasks to deal with compared to the tasks last meeting.

Yelo L:

RapiD makes editing easier for it can detect roads immediately, and you can use the feature so as not to map manually unlike the mapping on ideditor in NEPAL last cwts, we had to map everything manually which made it harder for us. Rapid Interprets roads and the tags but you still need to check if the interpretation is correct.

Richard R:

And for my opinion about usibg RapiD is that it makes me easy to find some patg ways that is hard to find but there some road that is detected but detected just a portion of it and some of the paths are unknown and some of the roads detected are not really roads.

Angelica L:

Para po sakin yung rapid napaka convenient po nya gamitin, madali po gamtin yung sa OSM pero mas madali po ito. Mas mabilis din po makapag identify dito dahil po na rerecognize na yung mga possible roads. Ang kailangan na lang po gawin is icheck kung may road po ba doon and identify pa po yung mga road na hindi nabasa mismo ng rapid,katulad po ng mga path ganon.

Giesel B

I noticed that in mapping rapid is much easier than the other. And I get it better than before.

Ljun

Rapid Tool is very helpful to identify roads slighty blurred because of pixelated imagery

Laurence B

RapiD is muschn easier and helpful in terms of identifying roads, path and other types of ways, but AI is not enough and still human intervention is needed tom make our mapping more accurate and relevant.

Patrick M

it was easily understandable for a beginner like me. I had a smooth work on my time when we had this activity, i enjoyed every minute of it. Pero, sir, nakakalito yung maraming sub-tasks within a single task. Sana katulad nung original na tasking manager, one area for one task lang.

Albert B

[rapid] it helps me to see easily the roads that are existed, and it helps me a lot to do my task very easy. It is easy because it identifies the error of the roads, and in my experience in using this rapid is exciting because it discover what i didn't see. Awesome Mapping

[maning]

Daghang salamat for the user feedback @govvin!

Let's proceed with the validation to evaluate the quality of edits.

[govvin]

Observations and notes fromCamiguin task validation

Better on-boarding best practices can contribute to better data by improving workflow.

• [ ] Add to on-boarding instructions: Toggle OSM data (ALT+W) and AI-detections (Shift+R) to clearly identify what needs to be validated, and what's needs to be drawn manually.
  • [ ] Probably due to Internet connectivity quality, after a task has loaded, the AI-detections will not immediately show-up, ask contributors to review for un-detected roads first, before validating AI detections.
  • [ ] Try to resolve detected issues (e.g. overlapping features, too many points) before saving
  • [ ] Encourage descriptive comments, especially if issues are encountered. Remind users to save work before they reach 40 changes.
• [ ] If possible, avoid creating tasks with multiple polygons
• [ ] Clarify that the task objective is to map all roads, either by validating the AI-detections and/or mapping identifiable road features. If neither is possible, the task is NOT complete.
  • [ ] If an AI-detection is incorrect (e.g. beach is detected as a road), then remove the feature. Do not leave it as it is. Optional: mapper can manually map the correct feature, and tag it appropriately.

[govvin]

Mapping instructions (and on-boarding during mapathons) should highlight important keyboard shortcuts in iD:

Shift+R - toggle detections Alt+W' - toggle OSM data W` - toggle OSM data fill

[maning]

Validation completed today.

Next actions

• Compile all validation notes and summarize here.

[maning]

Big thanks to @govvin and his CWTS++ students for testing the project! Below is a short summary of the evaluation (detailed comments for each task are available within the TM project).

Summary

• Highway coverage increased by 36% with the following. The top 5 highway tag that increased its coverage were: footway, service, path, track, unclassified.

highway | Pre-mapping (2019-05-23) in meters| Post-mapping (2019-09-02) in meters |---|---|---| footway | 977.00 | 4,364.70 path | 20,356.86 | 36,890.56 primary | 63,875.08 | 63,957.88 primary_link | 1,553.75 | 866.01 residential | 49,796.83 | 71,869.66 secondary | 17,440.70 | 17,397.70 service | 4,524.01 | 9,601.30 steps | 21.91 | 21.91 tertiary | 34,667.53 | 35,959.58 track | 34,021.27 | 57,977.43 unclassified | 51,986.94 | 80,918.91 living_street | | 88.75 **Total**| **279,221.88** | **379,914.39**

• Geometric quality of detection is very high and is comparable to the quality of human tracing. In most cases, there is no need to improve the shape of the line.
• There were several cases of false positive (FP) detection. Most of them were confusing even for humans. Examples of false detection were: dry stream/river bed, riprap, beach areas and airport runways.
• Not all missing roads were detected (false negatives) particularly those slightly obscured by vegetation.
• Using HRSL for identifying areas to map is effective and reduces the number of tasks needed to map.
• It is difficult to use multi-polygons for tasking. In some cases, mappers forget to review all detection if there are multiple polygons within a task.
• Mappers are sometimes confused with the correct highway tag to use, The difference between footway and path or track, unclassified and residential were the most confusing.
• When a false detection was removed/deleted during mapping, it remains in the task during validation. This resulted to several invalidation of tasks because the validator assumed that the detection was missed. This bug was already reported to RapiD developers.

Next actions

For the next mapping projects, we should:

• Use only square/grid tasks, mappers can miss mapping smaller polygon with the multipolygon approach.
• Provide detailed guidance on the difference between certain highway classes. We should update the PH tagging guidelines especially for lower class highways.
• During validation phase, validator should just map missed detection to avoid going through several cycles of re-mapping and validation. This particularly important due to the bug we found in RapiD.

[maning]

All good here, closing!