Support the Organised Editing Guidelines

pantierra commented 5 years ago

Recently, the OpenStreetMap Foundation, approved the Organised Editing Guidelines. These require reporting back to the OpenStreetMap (OSM) project, whenever editing efforts are coordinated.

The Tasking Manager (TM) is OpenStreetMap's favorite tool to managed organised editings, either in place (like mapathons) or virtually (e.g. remote mapping activities for humanitarian cause). A lot of the information required by the Organised Editing Guidelines is already present in the projects' descriptions in the TM. There the idea and feature request arises that the TM could (automatically) report to OpenStreetMap.

The preliminary investigation shall cover:

[x] Define contents from TM to be delivered to OSM; and eventual changes in the TM (e.g. new fields, etc.)
[x] The technical method of communicating.

pantierra commented 5 years ago

This issue has become a Google Summer of Code 2019 proposal.

pantierra commented 5 years ago

The Organised Editing Guidelines state:

Organised edits should have a Wiki page named [[Organised Editing/Activities/Name of the Activity]] for the particular activity, and record the page in the list under Organised Editing/Activities. (Sample Wiki pages are here)

I would estimate very roughly that there are ten-thousands of such organised editing activities happening per year with OSM. And I know explicitly about two TM instances that have around 1000 new projects every year. See here the stats for the HOT Tasking Manager:

Given that there are thousands of organised editing activities to be expected every year, I wonder whether this is still welcome in the OSM wiki to live one page each. Or whether it should be one large page? And of course we are interested in the right way this can be pushed automatically (if possible) to the wiki. I could also think about committing these reports into a repository that could be controlled by the OSM Foundation. But here it is crucial to get input from the OSM system administrators to get a conversation started on the best way we can facilitate the reporting of TM projects to OSM based on the guidelines. @Firefishy can you join the issue and give us your point of view, please? Thank you.

(cc: @grischard, @russdeffner)

russdeffner commented 5 years ago

@xamanu - there is this procedure doc I am working on, should be helpful to define what can/should come from TM: https://drive.google.com/open?id=1PUzp53wiu497eqoBiva48XvjB-Bb7Waj0jXCQgKqbqU

pierzen commented 5 years ago

With the advent of the MissingMaps project in 2014, the organizations that sponsor that project did not only organize mapathons but regularly advocated for groups around the world to organize mapathons. Since Nepal 2015, all the major OSM disaster Responses have suffered from massive participation of newbies with bad quality edits. The same with the various actions to help for humanitarian NGO's vaccination campaigns and other projects. While there was many reports about quality problems, we have not seen a major shift to better control these activities, train newbies and produce quality data.

The TM offers the possibility to escalate for any OSM Response with thousand of contributors participating. But if the only significant functionality is to distribute the mapping tasks, we start to have major problems with the thousand of One time newbies. We saw in the last few years an escalation of Organized mapping projects and Mapathons and the rising of quality problems.

The organizers of Disaster responses are loosing control with MissingMaps acting as a broker advocating for mapathons, and partner organizations inviting their branches in various countries to organize or promote Mapathons. We can say that we have a more complex Mapping network system where we dont know the significant actors that organize a mapathon and invite newbies, do or not do training, monitoring and quality control.

Coordinators of major OSM Responses like for the Ebola 2018 in DRC are faced with major quality problems with no sufficient control on the process of mapping. I reported on the HOT list The point on the OSM Response to the DR Congo Nord Kivu Ebola outbreak how in the middle of a major crisis, we had to redo from the start the Butembo city mapping. We then decided to limit the mapping to experienced mappers until satisfactory solutions are brought in for Mapathons and newbies contributions.

In one night, your project can be contaminated with a lot of quality problems spread over the project with insufficient monitoring tools for Overview / Analysis of the problems.

MissingMaps project have escalated significantly since the end of 2014, advocating to map for various projects related to humanitarian partners projects. Groups around the world are invited to Pop-Up with new mapathons and quality problems are poping-up at the same speed !

Badly, there is no real control, anybody can start a one night project without sufficient experience and have newbies without any experience to start mapping, this without good workflow to train and monitor newbies and assure data quality. This is what Pascal Neis classifies as «Hit and Run contributors». But in fact the organizers should take the responsability for the mapping with newbies. Since the start of the MissingMaps projects, we regularly reported quality problems but no significative actions have been taken to correct that while more and more mapathons are proposed.

As the TM is concerned, what Coordination role it could play other then distributing tasks ? Below are some ideas about various aspects of this.

In the last version of the TM, he functionality to follow each contributor was removed (previously, we could at least see rapidly which tasks a contributor completed). We should be able to see rapidly which tasks a contributor played any role in (edit / validate / reporting bad imagery, etc).

Not only the tasks with comment should be reported. If you want to follow closely, let's say each day or hour the last tasks edited / validated, you dont have access to such info.
-Statistics about edit and quality problems would help to monitor. We could have statistics of Ratio of Irregular buildings by contributor. This would help to spot rapidly such problems.
Validation should be restricted to experienced mappers unless a coordinator adds the osm_id in a blank_list.
iD should not be proposed as the validator by default in the popup.
the TM should remember the contributor preferences (ie. language, editor, etc)
Some thinking should be though about how to integrate mapathons organizers in the loop It should be looked if possible to add an organizations section with an id/Hasthtag for the organisation, description + coordinates of those that coordinate mapathons, dates/hour of mapathons and ways to add a hashtag to identify.

Clearly, we need a major shift for Mapping projects to be better monitored for quality.

ulshell commented 5 years ago

Hello I know python (flask ) and I did various projects in it. I am looking forward to hear back from you for gsoc 19

ethan-nelson commented 5 years ago

@ulshell you may want to join the GSOC channel on our Slack for a better place for inquiries regarding that program: http://slack.hotosm.org/

pantierra commented 5 years ago

So far i was not able to contact the infrastructure team of OSM in this regards. There are two Python libraries that allow interaction with the wiki: pywikiapi (simple) and pywikibot (more complex).

A good way of doing this, could be using wiki's data items to describe the reports. This can enable the Tasking Manager to manipulate data in the wiki when it is getting changed on the TM, and it may reduce the amount of duplicated text added to the wiki.

vkWeb commented 4 years ago

@xamanu Hi.

I'm interested in dealing with this issue for GSoC20. Right now I'm reading about OpenStreetMap and Tasking Manager.

Any advice / suggestions / directions to help me understand this issue clearly?

parth-gr commented 4 years ago

Hi.

I'm interested in dealing with this issue for GSoC20. Right now I'm reading about OpenStreetMap and Tasking Manager.

any input i can provide right now?

pantierra commented 4 years ago

We are going to apply for GSoC 2020 for this project. Looking forward to this. Stay tuned for more updates.

joaovitor3 commented 4 years ago

Introduction

Hello everyone!

I'm the GSoC student who is going to work on this topic during the project time. I wanted to update you on my first analysis regarding the data requested by the Organised Editing Guidelines and the information present in the Tasking Manager.

Please have a look and I appreciate any feedback from the community with any suggestions or advices.

Fields table

The table is divided according to the Organised Editing Compliance Procedures document. We splitted the fields into Present, Missing and Possible improvements as fields may be present or missing in the Tasking Manager, the Possible improvements is for fields that already present in Tasking Manager and a Possible improvement could be applied to it.

Field (OEG description)	Tasking Manager
Organisation and contact info
Description and link to organization	Present: Organisation name Organisation link Missing: Description of organisation
A way to contact the project manager or team	Present: Author OSM id
Project details
The goal and purpose of the activity	Present: Project Description Possible improvement: Explicit goal statement in project creation
The timeframe for the activity	Present: Project Creation date or publication date Missing: Suggested end date (time limit)
any non-standard tools and data sources used, and their usage conditions	Missing: Add new section in project creation
Links where the community can access any non-standard tools or data sources	Missing: Add new section in project creation
Standard changeset comment
Specific hashtag for tracking	Present: During project creation
Link to related organized editing activity	Present: Link to the project in TM
Team and User information
The accounts of participating persons that wish to be identified, with any details they wish to include	Present: OSM ID’s and profile information of mappers Possible Improvement: Allow people to select if they want to be identified Synchronization with OSM
if participants will receive training material or written instructions, a copy of, or link to, these materials: Links to organized editing organization(s) and activity(ies) on user profile sufficient training for project (i.e. local tagging schemas, etc.)	Present: Project instructions and per task instruction sections Link to learn section in OSM
if the success or performance of participants will be measured in any way, a description of the metrics used for this	Missing: Need to add a description of metrics in TM
Post project
Plans for a “post-event clean up” to validate edits, especially if the activity introduces new contributors to OpenStreetMap	Missing: Need to add a general description of quality assurance in TM Improvement: Add section in project creation to give specific information about validation processes
After the activity has completed, or at least once a month for ongoing efforts, a description of the results	Missing: Produce automated report
If an organisation has many different activities, it can create one page for the organisation that lists the common elements, and point to that from each activity page	Add a strategy for this case

Communication between Tasking Manager and OpenStreetMap

We started a repository to start to think around the comunication between Tasking Manager and OpenStreetMap. To kick this off, I implemented a small proof-of-concept service, that is able to get information from an application, like the Tasking Manager, and saves it.

In a first step it stores it as XML files in a git repository. I will change this to YAML in a bit.

The idea is to get the conversation started on what is the best way for the OpenStreetMap community and the Data Working Group to receive those automated input.

russdeffner commented 4 years ago

Great @joaovitor3 - good start. I was thinking, if you have not seen it, there is the Community Index; the iD editor uses it to suggest ways to connect with mappers in the area you just mapped. Maybe we could also tap into the index as a way to give notice, or for project creators to get the contacts listed in the area of their project(s). This could help satisfy 1.3.2.1 of the OEG which I don't see addressed in the table.

joaovitor3 commented 4 years ago

Thanks for the feedback @russdeffner Definitely, the topic 1.3.2.1 is not present in the table. I will validate the idea with @xamanu and as soon as we have new updates we will update this issue.

joaovitor3 commented 4 years ago

We have some news about the topic 1.3.2.1.

After meeting with @xamanu and @willemarcel we choose the mailing list as the best approach for this, although we still have to validate this and some other topics about the best approach of reporting data back to OSM with the Data Working Group.

After this validation, we will bring more updates.

Aadesh-Baral commented 2 years ago

Related to #3132

hotosm / tasking-manager