Broken transit relations in Phoenix metro

[hoream-telenav]

Greg from Phoenix got in touch through email and OSM message about broken transit relations in Phoenix caused by our road edits. This ticket tracks the fixing effort.

List of all bus relations in Arizona (not all are in Phoenix)

Spreadsheet with OSM Relation Analyzer links

OSMI link

[hoream-telenav]

From @jothirnadh on November 6, 2017 5:11

Data team at Mapbox also gone through this issue while trying to improve navigation data in OpenStreetMap. I did some research on the reason for causing this issue and wrote a diary post on this. Would be great to know your views as well in this case.

This is where we tracked our Portland transit lane relation breakage issue for further reference.

[hoream-telenav]

From @mihaii-telenav on December 7, 2017 9:0

We've extracted all the bus routes using this overpass query and we ran then through the Relation Analyzer and we proceeded to fix the ones that are broken. Here's an overview of the ones we found to be broken and were fixed.

Bus relations (the changesets mentioned below are the ones that completed the editing process on the relation):

• 6371050 fixed with changeset 54170090
• 6371049 fixed with changeset 54171334
• 4554056 fixed with changeset 54171930
• 4499237 fixed with changeset 54172256
• 4499232 fixed with changeset 54172395
• 4499242 fixed with changeset 54173720
• 4499243 fixed with changeset 54173947
• 4498658 fixed with changeset 54174052
• 4554055 fixed with changeset 54175233
• 4554048 fixed with changeset 54175649
• 4554059 fixed with changeset 54176234
• 6068158 fixed with changeset 54176432
• 6068157 fixed with changeset 54176558
• 6068165 fixed with changeset 54177132
• 4499192 fixed with changeset 54177597
• 4499193 fixed with changeset 54180582
• 4499238 fixed with changeset 54178369
• 6068166 fixed with changeset 54179003
• 4499213 fixed with changeset 54180582
• 4499146 fixed with changeset 54180089
• 4499140 fixed with changeset 54324604 - this relation has the platforms as polygons. Those were not modified.
• 4499141 fixed with changeset 54324604 - this relation has the platforms as polygons. Those were not modified.
• 4499227 fixed with changeset 54326032
• 4499248 fixed with changeset 54326522
• 4499249 fixed with changeset 54326995
• 4499174 fixed with changeset 54330108
• 4499223 fixed with changeset 54330417
• 4499205 fixed with changeset 54330855
• 4499191 fixed with changeset 54332811
• 4499194 fixed with changeset 54328839
• 4499222 fixed with changeset 54330855
• 4499190 fixed with changeset 54327928
• 4499164 fixed with changeset 54323008

2 bus relations were skipped because the route was going through a construction zone and the community left a comment on the relations.

• 6068162
• 4554058

Bike relations (the changesets mentioned below are the ones that completed the editing process on the relation):

• 5541148 fixed with changeset 54405402
• 5541152 fixed with changeset 54405807

4 bike relations were skipped because there wasn't enough information to update them. In order to be updated, local knowledge is the best option.

• 5541150
• 5509062
• 4020970 - community added a description tag on it: The Sun Circle Trail is an unfinished trail in Maricopa County that is enclosed by the Maricopa Trail.
• 4020964 - community added a description tag on it: The Maricopa Trail is an unfinished trail in Maricopa County that surrounds the cities and the Sun Circle Trail.

I've sent a message to Greg to ask for further info regarding the relations that we've skipped.

[hoream-telenav]

From @drkludge on December 10, 2017 20:51

Here are some random observations.

• I think one of best things that happened out of this problem was @mvexel introducing me to OSM Inspector Transport routes tool. One of the great strengths of this tool is the 24 hour turn around when you are trying to correct data or even implement data.
• I also want to thank both Mapbox data team's and Telenav data team's efforts in the metro Phoenix area. I am aware of the features that you both added to the map but this type of mapping is not always at the top of my mapping area of interest.
• I started fixing the routes because I could not remember if we cleaned up the relations after Mapbox data team's enhancement efforts. Thank you for your efforts. It seems like most of the routes you worked on were in the east valley where I had not started the repair work yet.
• The OSM Inspector wiki page has a great explanation of the validation. All the bus stops must be at the top of the relation table. You must add at least one stop at the top of the table and the route must be sorted for the route to go green. I looked at some other metro areas to see how they implemented the routes and understood this better.
• In some cases I would see a route go back to black the very next day after it had been corrected to green. All the data was still in the relation but two of the members were flipped after a way split or something. There were no visual holes in the route but one of the requirements for the route to go green is that the members sort from beginning to end without any gaps. The flipped members generated a gap in the sorting.
• I now see why an area relation is helpful for adding the stop position and platforms to a route. You can add both features to the route via the one area relation. Those have not been implemented yet verses just adding one stop/platform to make the route go green in the OSM Inspector tool.
• I think one of the contributing issues is that the USBR 90 Cycle route is 572 miles long. US 60 is almost 400 miles long. I had already corrected the USBR 90 last month. It looked like there was a conflict between an edit in Tuscon and another edit in Phoenix. @jothirnadh's blog post may explain some of the issue.
• I think another issue is the density of mappers. Metro Phoenix has a dynamic population. It is very hard to build a sustained group of mappers. Most of the time I can map for hours without saving and without conflict. However, when I know there is a large group of edits going on I hit the File > Update button before I attempt to save.
• A couple of times when I was fixing the relations, I managed to completely clear the relation members from the relation dialog. I could not repeat the steps. That does not explain how some routes were partially cleared. I have fixed the two AZ 51 Rapid bus routes but have not fixed both the AZ 51 highway routes.
• As with the Mapbox experience I grabbed a copy of Arizona from Fredrick's site. I picked pbf that was generated right before Telenav started to edit in Phoenix. I was able to repair many of the routes quickly by regexing the Osmosis route items back into the broken relation.
• I also found that forward and backward roles work for cycle routes but these role names throw many errors in the OSM Inspector tool. I also carefully regexed out those role names.

  4499233 512829821   W   forward 360
  4499233 241138363   W   forward 361
  4499233 527586905   W   forward 362
  4499233 436944787   W   forward 363
  4499233 408592759   W   forward 364
  4499233 436944788   W   forward 365
  4499233 408813167   W   forward 366
  4499233 408813155   W   forward 367
  4499233 408592761   W   forward 368

  I used either a vim regex pattern that transformed the Osmosis relation members back into an .osm file format or removed the role name if the route was mostly intact.

  :'a,'b s/^\([0-9]*\)^I\([0-9]*\)^I\(.\)^I\(.*\)^I.*$/    <member type='way' ref=
  '\2' role='' \/>/  

  The ^I is a tab character. The 'a and 'b are vim range markers. The markers were required to carefully make the changes in the selected area of the .osm file.

  :'a,'b s/'backward'/''/

  Note that you also have to make a change to the route in Josm before saving the file or add action='modify' to the relation tag.

  <relation id='4498932' action='modify' timestamp='2017-11-21T14:54:06Z' ...

• The use of the forward and backward roles was to assist with how some routes loop around and around. We have to add a way multiple times to a relation to make the route work properly and keep the bus exit on the correct side of the street. The relation editor marks these ways with red. These loops in routes became problematic especially when you try to use the OSM Inspector route validation tool. It was a major effort to sort the route segments for the route to go green. Since I was using master route relations, I added another set of routes to handle these loops as seen on the Bethany Home Road route. That way you can quickly sort the route with the relation editor and keep the pathological sorting of the loop to a smaller relation.
• From the looping relation idea, I though of splitting some of the longer routes. An example candidate is the Glendale Avenue and 24th Street. Dividing the route at 24th Street may help with some of the potential conflicting edits. However, the new child relation segments will still be very long.
• Route relations and relations in general provide a much needed tool for certain features that cannot be mapped by a node or way alone. However, the relations feel like they are fragile. I am not sure how to improve protection of the relations. I wonder how durable turn restrictions will be. The hidden nature of routing both visually and not knowing of the routing features may make these relations susceptible to breakage.