OSMPH / papercut_fix

Tracking data fixing requests for the OSMPH community
9 stars 2 forks source link

Fix overlapping/crossing buildings 🏛 🏠 🏘 🏙 [maproulette] #43

Closed maning closed 5 years ago

maning commented 6 years ago

_Continuing from https://github.com/OSMPH/papercut_fix/issues/13._ Update: new workflow.

screen shot 2018-03-30 at 09 16 58

10K overlapping/crossing buildings as of March, 2018

Using OSMLint, we found 10K overlapping buildings in the Philippines. 😱 I created a maproulette task to fix this. We need everyone to help!

  1. Go to: http://maproulette.org/mr3/challenge/2992/
  2. Grab a task. crossing-building
  3. Select all buildings. Click Validation (Shift + V).
  4. Fix all crossing/overlapping building detected by the validator.
  5. Changeset comment: Fixed crossing/overlapping buildings. https://github.com/OSMPH/papercut_fix/issues/43. Changeset source: OSMLint; Maproulette

Status

Date N (% change) Tasks (% change) ⏹ (1k items)
2018-03-24 10,513 3,560 ⏹⏹⏹⏹⏹⏹⏹⏹⏹⏹
2018-05-06 7,672 (-25.34 %) 3,299 (-7.33%) ⏹ ⏹⏹⏹⏹⏹⏹⏹
2018-05-28 7,619 (-0.69 %) 3,209 (-2.73%) ⏹⏹⏹⏹⏹⏹⏹⏹
govvin commented 6 years ago

Deja vu?

image

schadow1 commented 6 years ago

image Not Deja Vu @govvin Over 2.2 million buildings were mapped during the NOAH initiative back in 2016-2017 which comprises over 400% of the number of pre-existing buildings during that time. So it's probable and not surprising "many" of the overlapped buildings (31k or 1.3% of the buildings mapped) were mapped during the initiative. What I seem to not understand is how many times this was brought up on 2 opened issues over the 1 year period by you when it is already understandable where most of the issues came about, as per the number of updates made. I somewhat sometimes misconstrued this is a fault-finding task instead of a "fix". The priority of the project was to map buildings so that they can be intersected with hazard maps. Executive decisions and politics within the project during its course has limited the capacity of its manpower for thorough validations. Hence, as a respect to the people in the project, I am letting them have their say on this matter. I however do not know if they have already made a statement (publicly or privately) and what the statement was. I however believe (based on my personal monitoring) that a few people from NOAH has already assisted in fixing a small part of the 7k of the overlapped buildings in the past year. I however still could not comprehend why the source of the issue is brought up many times and what is the point.

govvin commented 6 years ago

"Fix" is the correct term. The issue is quality assurance, as it has been in the original ticket. The facts - just like the numbers you quoted, also show how these happened. These are not in dispute here.

It's been 10 months since these issues has been discovered, and partially fixed, but we don't see any more efforts from NOAH to complete this. The concern now is how to direct the limited resources of this community to clean up after this.

Thank you @schadow1 , for finally coming out to say what I surmise was the case, "limited capacity of its manpower for thorough validations."

Not Deja Vu @govvin Over 2.2 million buildings were mapped during the NOAH initiative back in 2016-2017 which comprises over 400% of the number of pre-existing buildings during that time. So it's probable and not surprising "many" of the overlapped buildings (31k or 1.3% of the buildings mapped) were mapped during the initiative. What I seem to not understand is how many times this was brought up on 2 opened issues over the 1 year period by you when it is already understandable where most of the issues came about, as per the number of updates made. I somewhat sometimes misconstrued this is a fault-finding task instead of a "fix". The priority of the project was to map buildings so that they can be intersected with hazard maps. Executive decisions and politics within the project during its course has limited the capacity of its manpower for thorough validations. Hence, as a respect to the people in the project, I am letting them have their say on this matter. I however do not know if they have already made a statement (publicly or privately) and what the statement was. I however believe (based on my personal monitoring) that a few people from NOAH has already assisted in fixing a small part of the 7k of the overlapped buildings in the past year. I however still could not comprehend why the source of the issue is brought up many times and what is the point.

schadow1 commented 6 years ago

@govvin Thanks for clarifying that it is the "fix" is what you are interested in, however that is not what I saw. But again it depends on how it is perceived, right?

In anyway, I had requested assistance for validations from the community over the months while the project was on-going upon seeing how the project aimed on a different focus. Just refer to the mailing list. So the issue regarding request for thorough validation is not new.

I am still thankful to the people who gave time to contribute on the project and hope this project provided many ways to benefit the country in terms of DRR. I hope the overlapped buildings, which may constitute many of the 1.3% of the mapping initiative results, be fixed with the organization's limited resources soon. Unfortunately, we can't just rely from that post-DOST project on this "fix" for many reasons.

"Fix" is the correct term. The issue is quality assurance, as it has been in the original ticket. The facts - just like the numbers you quoted, also show how these happened. These are not in dispute here.

It's been 10 months since these issues has been discovered, and partially fixed, but we don't see any more efforts from NOAH to complete this. The concern now is how to direct the limited resources of this community to clean up after this.

maning commented 6 years ago

🛑 The ticket is not intended to put blame on anyone. Please use this ticket to coordinate the fixing effort.

maning commented 6 years ago

Grid-based fixing instead of feature based (request for comments).

Whenever I fix an item, I always ran JOSM validator and fix adjacent overlapping/crossing buildings within my downloaded area. I propose we adopt this approach instead of a feature-based fixing.

Here's how it looks like in QGIS. screen shot 2018-04-06 at 06 09 29 z16 grid over MultiPoint detection

For a z16 grid (~600 x 600 meters), we have 3560 grids to cover the 10K crossing errors. This approach will significantly reduce items in our challenge and reflects the approach to fixing I (and perhaps others) do.

Pros

Cons

Next actions

maning commented 6 years ago

screen shot 2018-04-06 at 14 19 09 Thanks! Updated OP for the new workflow and challenge URL.

maning commented 6 years ago

Sorry, but I have to delete buildings traced like in this task: http://maproulette.org/mr3/challenge/2992/task/4659337

screen shot 2018-04-10 at 05 45 15

screen shot 2018-04-10 at 05 58 01

feyeandal commented 6 years ago

Hello! As we all know, the Catholic Relief Services (CRS) is currently implementing Project SUCCESS (Strengthening Urban Communities' Capacity to Endure Severe Shocks), directly supporting nine towns and cities in Metro Manila, Bulacan, and Rizal. The project aims to strengthen the capacity of local government units and vulnerable communities and enhance their resilience to floods and earthquakes. One of their main activities is to assist these LGUs to develop their disaster exposure database to help them identify and map the different structures that may be affected by natural hazards, so that they can better prepare and respond to these events. Last February, we conducted training and mapathon events to these cities to collect map information of buildings and other key elements on the ground. CRS formed a validation team within their organization, as well as UP Department of Geography to validate the data from the mapathon activities. I will be coordinating with these teams to help us fix the overlapping building issues, especially in Rizal area.

feyeandal commented 6 years ago

Hello!

Update and FYI: CRS agreed to help in validating the overlapping and crossing buildings in the areas covered by their project (some parts of Rizal, Malabon, and Marilao). They requested to just use their HOT Tasking Manager projects for validation instead of using the Maproulette task. The validation activities from the CRS team will be lead by @jdcayetano and @MARINOD.

Thanks!

maning commented 6 years ago

use their HOT Tasking Manager projects for validation instead of using the Maproulette task

Sounds good @feyeandal! Can you link here the TM projects?

govvin commented 6 years ago

@feyeandal , what's the latest with the validation effort of your project with CRS? cc: @jdcayetano @marinod

We still have these https://osm.org/go/4zhNQGxSE?m= :

image

feyeandal commented 6 years ago

Hello @govvin, the CRS team's validation is still on-going. They are also partnered with UP Department of Geography to fix the overlapping and crossing buildings in the CRS project sites.

Here are the following task numbers: Angono https://tasks.hotosm.org/project/4476 Cainta https://tasks.hotosm.org/project/4185 Taytay https://tasks.hotosm.org/project/4614 Marilao https://tasks.hotosm.org/project/4474 Malabon https://tasks.hotosm.org/project/4475

jdcayetano commented 6 years ago

Hello @govvin,

We are continuing our efforts until July/August so this might take some time as we don’t have enough validators.

Sent from iPhone

-- Janeen Kim Cayetano | GIS Analyst, Global Knowledge and Information Management (GKIM) | +63 917 8011 366<tel:+63%20917%208011%20366> | janeenkim.cayetano@crs.orgmailto:janeenkim.cayetano@crs.org | Skype: jdcayetano | 2<x-apple-data-detectors://3>F CBCP Bldg., 470 Gen. Luna St. Intramuros, Manila | crs.orghttp://www.crs.org/ | CRS Philippineshttp://www.facebook.com/CatholicReliefServicesPhilippines

On 31 May 2018, at 12:51 PM, Feye Andal notifications@github.com<mailto:notifications@github.com> wrote:

Hello @govvinhttps://github.com/govvin, the CRS team's validation is still on-going. They are also partnered with UP Department of Geography to fix the overlapping and crossing buildings in the CRS project sites.

Here are the following task numbers: Angono 4476 Cainta 4185 Taytay 4614 Marilao 4474 Malabon 4475

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/OSMPH/papercut_fix/issues/43#issuecomment-393401468, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcZAKZjr5g2szuv0VKfTN82fSm535lklks5t33a_gaJpZM4TBRGh.

govvin commented 6 years ago

@feyeandal @jdcayetano

Thank you for the quick heads-up. We're also still waiting to see the public project page, along with the list of editors and validators.

feyeandal commented 6 years ago

Hello, sorry for getting back late.

Here are the list of Youth mappers for each project site:

Taytay, Rizal AdlerBBEMNHS Jz_Mzlng nomadiko Joshua Abello EdjinJerney YvesTDM ShadowPillow markyc50 jang2x8 Elay Canaria Arleane Cariso Daniel Dave Abe Lavarez MISMNHS Efjay MISMNHS Davey MISMNHS Kathy MISMNHS Krisha MISMNHS Valerie MISMNHS Regine MISMNHS Christian MISMNHS Ivan MISMNHS Caila MISMNHS Del

Marilao, Bulacan SMSMSwin JustArns Ryanne0612 djjorge Nicah Fatima ReignYer PNHSJMPQI PNHSSRMP PNHSCECI PNHSMGS PNHSMFG PNHSLJSR PNHSKCMG PNHSCPM PNHSJCP PNHSRRC PNHSSCVV PNHSAAA PNHSJJFJ

Cainta, Rizal jiaisabel camille rose Ernan joshua buen23 peneyragresola

Malabon Steeven14 camille mnlpz CatherineFaith Eman Santos BarbstheGreat jeremyyotch EuniceCesniros13 ListraComia ErynRae14 RheaMae JulieAnne04 SophiaBianca CieloJhane FrenshieMae ElaineBorja2004 YasmienNicolle AngelRonquillo JencellSahagun AnaMarie03 Thund3rSt0rm JeremiBuitizon MarniVidal MikaelaManuel Krisna Angela Papa Khriz Aldridge LazaroChristelle MarkClarenceAguilon EmanuelJess MichaelJay RoviDeGuzman00 IVANJIMENEZ DennisLacson01 StevenTarcena HowardVelasco EdrianVizmanos NadineCanasta12

Angono, Rizal ianaguinaldo Laurence Royol geronimo bulandra bryanken

Will get back to you for the list of validators.

govvin commented 6 years ago

@feyeandal Thank you. In the future, please update this wiki page for updates regarding this project: https://wiki.openstreetmap.org/wiki/Project_SUCCESS

I took the list you posted earlier and added that as the initial entry for the page. Kindly add the other project details, when you can.

maning commented 5 years ago

Just visited the maproulette challenge and it was now complete!

Screenshot 2019-05-24 22 00 43