Adding support for other cities

[taybin]

What's involved with this? It's not just converting the OSM data right? Are locations hardcoded in somewhere? My city has terrible street traffic. I'd love to modify this to support other cities easily.

[dabreegster]

The short answer to get something going right now: https://github.com/dabreegster/abstreet/blob/master/docs/new_city.md

The longer answer:

• The ./import.sh and ./prelease.sh scripts (and now data/grab_seed_data.sh too) are meant to be able to fetch all of the input data needed in a reproducible way. They're just grabbing Seattle stuff right now. The proper answer is to make the map building pipeline generalized, probably config-driven. So ultimately somebody could come in and say './import.sh seattle_area' or london_area.
• Some of the side input files into https://github.com/dabreegster/abstreet/blob/master/convert_osm/src/main.rs are specific to Seattle. OSM and GTFS (transit feed) are standard, but the parking and sidewalk bits are shapefiles from the local GIS agency. Ideally all the data would live in OSM, but for Seattle (and most places) that's not true right now. So we'd have to add in support for an on- and off-street parking source for your city. (Does it have one? http://data-seattlecitygis.opendata.arcgis.com/datasets/blockface is the example for Seattle)
• Demand data (where do trips start and end, what time, what mode) is important if you want to test against realistic conditions. There's some code in sim/src/make/Scenario to uniformly pick source/destination buildings at random as a fallback, but the results are bad.
• You'll probably hit geometry problems inferring road segments and intersections from the OSM centerlines -- and from trying out other cities previously, there might be new types of problems. The map_editor crate lets you kind of manually edit the weirdest geometry issues. Ideally the map conversion process just becomes more robust generally, but there are some hard problems buried in there that I've been stuck on for a long time.

https://github.com/dabreegster/abstreet/blob/master/docs/articles/map/article.md and https://docs.google.com/presentation/d/1cF7qFtjAzkXL_r62CjxBvgQnLvuQ9I2WTE2iX_5tMCY/edit?usp=sharing give a good idea of what map importing involves.

This issue is a long-term priority, but I can't drive it anytime soon (there are too many other pieces that I want to polish first). If you're interested in contributing, I will do whatever I can to help you along (except leading the effort).

[dabreegster]

Other random things I forgot, both easy to generalize:

• https://github.com/dabreegster/abstreet/blob/1fc7a7388224e346ea12977e13cbf0305e6ae555/kml/src/main.rs#L12 hardcodes some GPS bounds
• RawMap -> Map uses MapFixes (produced by map_editor). We'd need to split the directory and store fixes per area and update a few places

[taybin]

Well, I grabbed my state off of open street maps and got a crude polygon for my city Then took a closer look and saw you were grabbing transit data as well. That'll take a little more research on my end to see what my area supports. In the meantime, going to see what condition the intersections and such are in first. Thanks, I'll let you know if I have more questions. Double thanks for the fast reply!

[dabreegster]

Most of the flags to convert_osm are optional, you don't need GTFS upfront. cargo run --release -- \ --osm=../data/input/osm/your_city.osm \ --output=../data/input/raw_maps/your_city.bin

[taybin]

One suggestion is to package the data under data/input by the map, so that it'd look something like data/input/seattle/osm and data/input/seattle/poly. This would create little map bundles instead of having different files for the same neighborhood in different locations.

[yasomatinandana]

Was trying to add a map for my local CBD (Perth, Australia) and it's throwing an error in the final 'precompute' command.

Specifically, it is saying thread 'main' panicked at 'What's turn restriction reverse?' in map_model/src/lane.rs:236:26 as well as 'dropping Timer during InitialMap to half of Map, due to panic?'

[dabreegster]

https://wiki.openstreetmap.org/wiki/Key:turn says 'reverse' means U-turns only. I'm not modelling U-turns properly yet; right now they get mapped to left turns usually. So sync past the above commit and try again -- that intersection with the restriction might be weird, but it should at least unblock you a bit.

[yasomatinandana]

There appears to be some difficulty with the conversion of certain intersections (see two screenshots attached). Would you be able to point me in the right direction of trying to fix this or are the intersections here in Perth too abnormal for the convert_osm to properly draw?

[yasomatinandana]

for further information, here is the sat view of the same intersection

[dabreegster]

In short, this is a hard problem that I've struggled with myself for a very long time. I'm certain there are simpler approaches to solving this that a fresh perspective would bring. Start with slides 7-20 of https://docs.google.com/presentation/d/1cF7qFtjAzkXL_r62CjxBvgQnLvuQ9I2WTE2iX_5tMCY/edit?usp=sharing. https://github.com/dabreegster/abstreet/blob/master/docs/articles/map/article.md has lots more detail with code references. https://github.com/dabreegster/abstreet/blob/master/map_model/src/make/initial/geometry.rs is one of the main places. If you have a general idea how to make intersection polygons from OSM centerlines and widths, but don't want to wade through all of the code there now, let me know your idea, and I can try implementing it.

One of the first things to check is that each road segment has the correct number of lanes in reasonable places. Looks like the sidewalks on the north half of Victoria Ave are on the incorrect side, because I've hardcoded driving on the right side of the road (and so for one-way streets, only putting a sidewalk on the right side). Look for the comment "Reverse points for British-style driving" in map.rs and and try unshifted_pts.reversed() a few lines down. I'll think through the right place to plumb through a parameter to specify driving on the right/left.

If you send the .osm file you're feeding in, I can spend some time tomorrow working on the right/left issue and seeing if I can fix a few more things.

[yasomatinandana]

Here is a G-Drive link for both the CBD and overall Perth osm (I was using the Perth CBD in the example above) https://drive.google.com/drive/folders/1tVv1NLy-G0FasyCboeP8i-vIix8a0VC1?usp=sharing

With regards to how to make intersection polygons from OSM data, I am just a grad electrical engineer and I have found my self in the traffic signals management space with a side interest in learning how to simulate/model. I am eager to learn/help in whatever way I can :)

[dabreegster]

Thanks! I'll try converting tomorrow and work on some bug fixes. I'd love any ideas you have about the intersection polygon problem or others. If you have a chance to try out editing a traffic signal, any thoughts about the user interface?

[yasomatinandana]

Awesome! I'll let you know my thoughts once I have poked around a bit more. I almost forgot, with the Perth osm there is an additional OSM highway tag that isn't previously defined in your code. I added the line corridor=0 under map_model/src/road.rs around line 427

[dabreegster]

Support for driving on the left side of the road is actually a fair bit harder than I anticipated. I'm about halfway through the changes needed. This particular intersection is looking much better. The biggest complication is that both roads have some kind of median, so they're mapped as separate one-ways in OSM. Coming up with good geometry for this sort of thing has always been a challenge; https://www.openstreetmap.org/node/29464223 is the equivalent example in Seattle.

A big barrier to running simulations in any city is getting data about where parking is available, both on-street and in lots/garages. Looks like there's no https://wiki.openstreetmap.org/wiki/Key:parking:lane tagged in Perth. For Seattle, I have an extra dataset from the local GIS agency and some heuristics based on OSM highway types. Are there any typical patterns for on-street parking in Perth?

[yasomatinandana]

Went down a bit of a rabbit-hole, but it appears Australia's Federal, State and Local government have this handy-dandy online map/data visualisation tool at https://nationalmap.gov.au/ and for the additional parking bay/lot data you asked about above you would need to get data sets COP-004 and COP-031

[dabreegster]

Thanks! There's already some tooling to explore how KML data is snapped to roads; I'll update it to handle GeoJSON too.

More generally, now could be a time when I spend some effort doing the work at the beginning of this bug. I'd like to start including more cities by default in A/B Street; I just don't have time to do all the leg-work of making sure each place works well. If you're willing to hunt down data sources like trip demand data, GTFS, elevation, etc for Perth, then I can start generalizing the import pipeline.

[dabreegster]

There's now much better support for this, see https://github.com/dabreegster/abstreet/blob/master/docs/new_city.md. I'm tracking remaining work in https://github.com/dabreegster/abstreet/projects/1. If you have specific issues importing a city or just want to see one in particular added, I vote we split into more specific issues