Importing OSM data for London

[georgio8]

Hi, your progress with the LTN Tool is exciting and it's already potentially useful. London UK is a focus of a lot of effort in planning LTNs.

As you may know, London is administered as 33 boroughs that act as separate cities for most road planning. My own borough is Camden https://www.openstreetmap.org/#map=13/51.5428/-0.1809. It would be great to get the OSM data for Camden imported to AB Street. I'd be happy to designate the area or to download the relevant section of the OSM database.

Once knowledge of the LTN Tool spreads I imagine there will be demands from quite a few other London boroughs too. Does it make sense to import the OSM data for all of Greater London (http://download.geofabrik.de/europe/great-britain/england/greater-london.html)?

[dabreegster]

Hi, thanks for trying out the early prototype! It would be great if you could draw the boundary of Camden with http://geojson.io/#map=13/51.5428/-0.1594. This might not just be the official borough boundary -- for studying LTNs, maybe you need to include more of surrounding areas too, for cases of a neighborhood spanning boroughs. So it takes local knowledge, which I don't have for London... yet.

I am very interested in importing all of the boroughs, and manually deciding boundaries seems tedious. I found https://skgrange.github.io/data.html, and the size of most boundaries looks reasonable (denser and physically larger areas mean bigger imported files, and the tool being slower at initially loading). Kind of awkwardly, the river is clipped out, which is an example of the interesting transportation study area not matching up with administrative boundaries:

But it could be a start. Let me know if you're in favor of starting with the official boundaries or not, and if you have a version of Camden that differs from this one.

I'm hoping to flesh out the LTN prototype in the next few weeks. Any particular requests or ideas what it should do?

[georgio8]

Hi, thanks for all the amazing work you're doing and for engaging about getting the London boroughs loaded.

I think you're right that we should aim to load each London borough separately, with a generous margin to enable study of overflowing LTN schemes. Here is a Gist with the geojson for the official Camden boundary and a bounding polygon providing a suitable margin: https://gist.github.com/georgio8/dd00872207e2210c0cea6d36f14709c6 It wouldn't be hard to do the same for all 33 boroughs.

About your future plans for the tool, we should engage with other interested parties including Brian D and @Robinlovelace in this discussion, but here are my initial thoughts:

• it would be great to add modelling of the likely routes of the traffic displaced when modal filters are installed. e.g. the user indicates some a-b routes through the LTN area starting maybe a km outside the area and the tool reroutes them when a filter is installed.
• even better would be to add traffic counts to the a-b routes and the tool would re-allocate the numbers to new routes when filters are installed.

This is all going to be very useful for our campaigning in London. At an appropriate point in the development I will introduce a couple of the professional highway planners we interact with Camden who draw up the plans for future LTNs and ask them for comments and suggestions.

Incidentally, here is a map we have produced showing the existing LTNs in Camden with a legend showing summaries of their areas and the rate of progress. It was produced with a SheetMapper tool that I hacked up using Mapbox and the Google Sheets API. We have used the same tool to produce several maps recording progress with cycling infrastructure.

[dabreegster]

Here is a Gist with the geojson for the official Camden boundary and a bounding polygon providing a suitable margin

Thanks! The result is one the larger end (60MB uncompressed, 22MB compressed), but it should still load comfortably on web. I'll post a link in a few hours when the new release is built.

we should engage with other interested parties including Brian D and @Robinlovelace in this discussion

Both are already involved. :)

the user indicates some a-b routes through the LTN area starting maybe a km outside the area and the tool reroutes them when a filter is installed.

Hmm, good idea. I've been thinking of detecting shortcuts just between two points along the perimeter of a neighborhood, like: with the motivation for the shortcut being to avoid traffic on the main roads. But that does feel like too small of a view to understand the actual motivation. If we can find or generate some origin/destination pairs for driving trips, we can start by finding the ones passing through the neighborhood, then calculate the likely detours.

even better would be to add traffic counts to the a-b routes and the tool would re-allocate the numbers to new routes when filters are installed.

We've tried something like this before in the traffic simulator. An example of making two roads between Poundbury and Dorchester one-way: The simulation predicts that the north/south road with the tooltip will see more vehicle traffic, as some drivers use it to switch directions. We can do something even simpler, without simulation, in the LTN tool if we have (or can generate) desire lines with counts for vehicle trips.

Is the motivation for this feature to explore/address concerns about what nearby streets might see "spillover" effects from a new modal filter? I feel like it'd also be important to have some tuneable controls to show "traffic evaporation" as people decide to forgo that driving trip altogether.

here is a map we have produced showing the existing LTNs in Camden

Ah thanks, these are great to reference! The A/B Street importer doesn't yet understand the many ways modal filters can be tagged in OpenStreetMap, so I'll check against this map and make sure the present situation is detected correctly.

[dabreegster]

http://play.abstreet.org/0.2.64/abstreet.html?--ltn&system/gb/london/maps/camden.bin Here's the Camden area. Entire thing loads in under 10s on my machine, which isn't bad for the browser version

[georgio8]

On 31 Oct 2021, at 20:48, Dustin Carlino @.***> wrote:

with the motivation for the shortcut being to avoid traffic on the main roads. But that does feel like too small of a view to understand the actual motivation. If we can find or generate some origin/destination pairs for driving trips, we can start by finding the ones passing through the neighborhood, then calculate the likely detours.

We have been asked to produce that kind of data by highway officers for use in their proposals. even better would be to add traffic counts to the a-b routes and the tool would re-allocate the numbers to new routes when filters are installed.

We've tried something like this before in the traffic simulator. An example of making two roads between Poundbury and Dorchester one-way:

The simulation predicts that the north/south road with the tooltip will see more vehicle traffic, as some drivers use it to switch directions. We can do something even simpler, without simulation, in the LTN tool if we have (or can generate) desire lines with counts for vehicle trips.

Some of the areas for consideration as LTNs in Camden have detailed traffic counts. This one has a 24-hour origin-destination survey of motor traffic https://camdencyclists.org.uk/campaigns-2/visualisations-of-camdens-kentish-town-o-and-d-survey/. Is the motivation for this feature to explore/address concerns about what nearby streets might see "spillover" effects from a new modal filter? I feel like it'd also be important to have some tuneable controls to show "traffic evaporation" as people decide to forgo that driving trip altogether.

Yes. Our highway officers, who are very supportive of reallocating road space to cycling, do cite traffic evaporation in their proposal docs, but only qualitatively. I don’t know of any research that would enable the quantitative prediction of evaporation.

A couple more queries to you: My partner Jean and I have been exploring the LTN tool on some neighbourhoods we’re familiar with starting with the ‘Browse Neighbourhoods’ screen. The neighbourhoods defined by the tool very often seem well-chosen, but we’d be very interested to know your algorithm for defining a Neighbourhood.

On some occasions, we would have liked to be able to extend a Neighbourhood to include another street, or even to merge two Neighbourhoods. But if you would tell us your Neighbourhood algorithm we can comment further.

You will be aware of the several types of filter that are used. We have examples of all these in Camden: bollard closure diagonal bollarded closure at a 4-way junction one-way closures

I expect there are more. Brian D can probably enlighten us.

[dabreegster]

This one has a 24-hour origin-destination survey of motor traffic

The data and visualizations here are awesome! I will think about how we can import this and make use of it for calculating possible detours.

your algorithm for defining a Neighbourhood.

Currently very simple. Start at any "minor" road and "flood" outwards until you hit a "major" road. Major roads are defined as motorways, trunks, primary, secondary, and tertiary roads in the OpenStreetMap classification. Minor roads are everything else. Depending on how current modal filters are tagged in OSM, sometimes one existing LTN will get detected as multiple.

On some occasions, we would have liked to be able to extend a Neighbourhood to include another street, or even to merge two Neighbourhoods.

Noted! I have some in-progress work to trace around a single "block" and merge adjacent ones. Once I get it working, I think I can let you draw boundaries however you like. I'm also thinking it's going to be important to let people mark a shortcut that they observe in real life or want to talk about, and not just browse through the list that the tool detects.

I hope to have some progress by next Sunday's build. In the meantime, if you know any software developers interested in the Rust language and this project, I'd love to build out the team working on this a bit more.

[georgio8]

The data and visualizations here are awesome! I will think about how we can import this and make use of it for calculating possible detours.

I wouldn’t put too much effort into that because such surveys are too expensive for regular use. But it’s interesting as an illustration of what can be done with good data. That survey is already outdated by a couple of filters that have been put in since it was done.

We are finding some issues with the LTN tool in the same neighbourhood (it’s where I live). Some missing rat runs and many false rat runs. The latter are not much of an issue and may be down to poor tagging in OSM, but the missing rat runs may be due to algorithm problems and should be discussed.

I’ll send more detail about those issues soon.

[dabreegster]

the missing rat runs may be due to algorithm problems and should be discussed.

If you send along an image of a missing example, it'd be useful. But I wouldn't spend much effort yet on looking at results from the tool. I've literally spent only a few hours putting together the prototype so far, and the rat run detection is almost deliberately bad at this point. I hope to properly focus on it in the next few weeks.

[georgio8]

False rat-runs

your algorithm for defining a Neighbourhood.

Currently very simple. Start at any "minor" road and "flood" outwards until you hit a "major" road. Major roads are defined as motorways, trunks, primary, secondary, and tertiary roads in the OpenStreetMap classification. Minor roads are everything else. Depending on how current modal filters are tagged in OSM, sometimes one existing LTN will get detected as multiple.

Yes, the algorithm’s handling of modal filters is problematic. The perimeter for this neighbourhood has been set at the modal filters indicated, and that has resulted in a lot of false rat runs ending at the filters (like the one included in this screen shot), when in fact the filters are effective in preventing rat runs. Even one road that is filtered by a bollard (Sandall Road) is shown as a rat run.The algorithm probably ought to ignore filters on minor roads so that neighbourhoods will be bounded by major roads. Another issue is that several false rat runs are shown that use private roads that vehicles can’t enter. Some of them are gated and others just signed.

Missing rat runs:

the missing rat runs may be due to algorithm problems and should be discussed.

If you send along an image of a missing example, it'd be useful. But I wouldn't spend much effort yet on looking at results from the tool. I've literally spent only a few hours putting together the prototype so far, and the rat run detection is almost deliberately bad at this point. I hope to properly focus on it in the next few weeks.

Several important rat runs in this neighbourhood (in pink and mauve) haven’t been identified by the tool:

[georgio8]

[Repeating the last post with images included] False rat-runs your algorithm for defining a Neighbourhood.

Currently very simple. Start at any "minor" road and "flood" outwards until you hit a "major" road. Major roads are defined as motorways, trunks, primary, secondary, and tertiary roads in the OpenStreetMap classification. Minor roads are everything else. Depending on how current modal filters are tagged in OSM, sometimes one existing LTN will get detected as multiple.

Yes, the algorithm’s handling of modal filters is problematic. The perimeter for this neighbourhood has been set at the modal filters indicated, and that has resulted in a lot of false rat runs ending at the filters (like the one included in this screen shot), when in fact the filters are effective in preventing rat runs. Even one road that is filtered by a bollard (Sandall Road) is shown as a rat run.The algorithm probably ought to ignore filters on minor roads so that neighbourhoods will be bounded by major roads. Another issue is that several false rat runs are shown that use private roads that vehicles can’t enter. Some of them are gated and others just signed.

Missing rat runs: the missing rat runs may be due to algorithm problems and should be discussed.

If you send along an image of a missing example, it'd be useful. But I wouldn't spend much effort yet on looking at results from the tool. I've literally spent only a few hours putting together the prototype so far, and the rat run detection is almost deliberately bad at this point. I hope to properly focus on it in the next few weeks.

Several important rat runs in this neighbourhood (in pink and mauve) haven’t been identified by the tool:

[dabreegster]

Thank you for the examples! Making some progress here. Now the tool shows contiguous areas reachable from the perimeter of the neighborhood in the same color. Before any interventions, the fact that all the roads are blue means all 3 of the example rat runs you drew are detected: I can place one filter at Busby Place and stop the east/west movement:

There's still lots of work to handle private roads, gates and barriers in OpenStreetMap, one-ways, etc. Hopefully a bit more will be working by this weekend.

[georgio8]

Thank you for the examples! Making some progress here. Now the tool shows contiguous areas reachable from the perimeter of the neighborhood in the same color. Before any interventions, the fact that all the roads are blue means all 3 of the example rat runs you drew are detected: I can place one filter at Busby Place and stop the east/west movement:

This is impressive progress. It’s great to be working with you on this - the LTN tool is potential fulfilment of a similar idea I have been nurturing for a couple of years - but didn’t have the time/coding skill to proceed with it.

We have watched and interacted with our local highways planners while they were trying to define ‘watertight’ LTN schemes and seen how difficult it can be. Added to the pure logic of making it watertight is the need to ensure that traffic can still move with a reasonable time penalty, hence my suggestion about analysing A-B routing that begin and end outside the LTN.

Incidentally, you might be interested in this small tool https://traffictool.rgp.me.uk/ that I developed with another cycling buddy. It piggy-backs on the Google Directions API to plot predicted journey times over a period.

There's still lots of work to handle private roads, gates and barriers in OpenStreetMap, one-ways, etc. Hopefully a bit more will be working by this weekend.

I’ve noticed that there is a lack of OSM tags for traffic filters. They always seem to involve a section of closed or one-way street with cycle tracks alongside. A single tag would do the job much more simply and clearly and be easier it interpret in apps like yours.

[dabreegster]

Here's the new version: http://play.abstreet.org/0.2.65/abstreet.html?--ltn&../data/system/gb/london/maps/camden.bin There are many issues I'm still working through -- don't expect the gated roads to be handled correctly yet. Seeing individual rat runs is disabled, because it's still far from working properly. But a neighborhood is split into different "traffic cells" (I think I've seen that term used?) and can be drawn two different ways.

the LTN tool is potential fulfilment of a similar idea I have been nurturing for a couple of years - but didn’t have the time/coding skill to proceed with it.

I have the opposite problem -- I enjoy coding stuff like this, but have a hard time coming up with the ideas that can actually be useful from an advocacy or planning perspective. I'm looking forward to working more closely with the London community in a few weeks...

I’ve noticed that there is a lack of OSM tags for traffic filters. They always seem to involve a section of closed or one-way street with cycle tracks alongside.

I agree the schemas currently used can feel unnecessarily complex. I haven't dove too far into this yet. I'm trying to first model all the different types of filters -- rendering them, adding or editing them, and affecting routing -- without rigorously importing existing filters from OSM. Once I have things solid from the tool's end, then the diversity of OSM tagging can be handled.

I swear I remember seeing a guide to tagging modal filters from CycleStreets, but I can't find it at the moment. https://www.cyclestreets.org/news/2021/07/25/mapping-ltns/ is related.

[dabreegster]

Also, if you have any time in the next few weeks for a 30-60 min usability study by videocall, it'd be massively useful to our design process to get more rapid feedback. Feel free to email me (dabreegster@gmail.com) to schedule if so.

[Robinlovelace]

Following this thread with interest, great to see a clear use case supported by substantial real world knowledge. I have had a look at the Camden example and it's looking great to me.

Observations (which are not new, in support of previous conversations):

• It's amazing how 'non-linear' the process is: you can block off several roads with little impact on watertightness and then suddenly one more in a seemingly unimportant link can create a large LTN, other tools will struggle to capture this.
• A functional (and aesthetic) improvement would be to allow modal filters to be placed at cross roads, with options enabling them to go diagonally to block certain turns. I have an example of this close to my house illustrated in another issue, it's highly effective.
• In terms of aesthetics, the LTN zones change pretty frequently with each click. This is a very minor comment in the scheme of things.

[georgio8]

It’s looking good.

And good to have Robin following. I agree with him that ‘diagonal filters’ can be very useful and they are fairly frequently deployed in London. I’ll point out a couple in Camden in a future post. This implies the need for the tool to provide a way to install more than one type of filter. There are probably other categories to should be distinguished.

In this post we’ll concentrate on giving you some further detailed feedback on the neighbourhood identification.

My partner Jean Dollimore, who is another active cycling campaigner with even better local knowledge than me, and I have spent some time pouring over the neighbourhoods that the tool identifies in Camden. The majority of them are impressively convincing and correspond to areas we and/or Camden transport planners have identified as LTN candidates.

However we have identified a few algorithm issues emerging from a minority of them and have written about them below. In our examples we have used road names and other names to identify areas - let me know if they aren’t easy for you to locate.

1. Parks (green spaces) should generally not be included in neighbourhoods. e.g. the map underneath should show through. This is always going to be true for the large ones e.g. bigger than about 3-4 hectares. Examples: Regents Park is identified correctly. But the following are shown as neighbourhoods: Hampstead Heath (227 hectares) Primrose Hill (25 hectares) Highgate cemeteries (18 hectares) Waterlow Park (11 hectares) St Pancras Gardens (6 hectares) Other parks that are on the edge of a neighbourhood should not be included within it. That is, the boundary of the cell should be drawn round it. Examples are Corams Fields, Tavistock Square and Gorden Square. Whereas small parks such as Kilburn Grange can be enclosed within a neighbourhood.

2. Blank areas: Some areas are not described as neighbourhoods but ought to be. Listed but without any theory as to why

   • area south of Euston Road around Fitzroy Square with Tottenham Court Road on eastern border and Cleveland Street on western border.
   • another area south of Euston Road by Mabledon Place
   • area bounded by Guilford St, Gray's Inn Road, Mount Pleasant and Farringdon Road

Other blank areas we agree with:

• University College (by Gower Street)
• The British Museum
• Gray’s Inn
• Euston Station (mainline train station)
• The British Library
• Maiden Lane (a self-contained housing estate)

Areas that we think should be blank but are not:

• St Pancras Station (mainline train station)
• Kings Cross Station (mainline train station)

3. Boundaries:

   • you already mentioned main roads (i.e. primary, secondary or tertiary). We don’t want to see any residential or unclassified roads on boundaries as we want to stop rat running through.
   • When we can’t find a nearby main road, we use rivers, canals and railway lines (when they are at ground level) Here are a couple of examples where the boundaries are not main roads:
   • The neighbourhood on the west side of Camden High Street has Jamestown Road as a northern boundary and Oval Road as a western boundary; we think they should be replaced the canal and railway respectively
   • The neighbourhood bounded by Prince of Wales Road, Kentish Town Road, Hawley Road and Camden High Street has a a residential road (Harmood Street) as the western boundary. The boundary should be further west on Ferdinand Street
   • The neighbourhood bounded by York Way, Copenhagen Street and Caledonian Road. The southern boundary is on an unclassified road. We think it should move to the canal towpath as in the rest of this edge.
   • The neighbourhood around Camley Street south of Agar Grove. Camley street is unclassified therefore the boundary should be railway on east and canal on the west.

4. Neighbourhoods that are severed e.g by a railway, a river or a chasm (we’re not sure about this logic as a rule but we don’t accept them) example neighbourhoods:

   • bounded by Kilburn High Road, Iverson Road, West End Lane and Quex Road. The area north of the railway line is not connected even by a footpath to the rest of the cell. The next choice would be Netherwood Street (which is residential) so you would need to use the railway as a boundary
   • bounded by Kilburn High Road, Maygrove Road, Mill Lane and West End Lane. This is severed by by the Midland Main line railway. We believe that the railway line should divide the area into two separate neighbourhoods.

• For discussion: Perhaps a high-level definition of a potential LTN is required, something like: An area containing only streets that can't usefully form part of any route between points outside the area. This definition allows an area to contain streets that aren’t connected for internal trips by any mode. So we could add: All the streets should be connected, at least for travel on foot or by bicycle.

On 7 Nov 2021, at 21:03, Dustin Carlino @.***> wrote:

Here's the new version: http://play.abstreet.org/0.2.65/abstreet.html?--ltn&../data/system/gb/london/maps/camden.bin http://play.abstreet.org/0.2.65/abstreet.html?--ltn&../data/system/gb/london/maps/camden.bin There are many issues I'm still working through -- don't expect the gated roads to be handled correctly yet. Seeing individual rat runs is disabled, because it's still far from working properly. But a neighborhood is split into different "traffic cells" (I think I've seen that term used?) and can be drawn two different ways.

the LTN tool is potential fulfilment of a similar idea I have been nurturing for a couple of years - but didn’t have the time/coding skill to proceed with it.

I have the opposite problem -- I enjoy coding stuff like this, but have a hard time coming up with the ideas that can actually be useful from an advocacy or planning perspective. I'm looking forward to working more closely with the London community in a few weeks...

I’ve noticed that there is a lack of OSM tags for traffic filters. They always seem to involve a section of closed or one-way street with cycle tracks alongside.

I agree the schemas currently used can feel unnecessarily complex. I haven't dove too far into this yet. I'm trying to first model all the different types of filters -- rendering them, adding or editing them, and affecting routing -- without rigorously importing existing filters from OSM. Once I have things solid from the tool's end, then the diversity of OSM tagging can be handled.

I swear I remember seeing a guide to tagging modal filters from CycleStreets, but I can't find it at the moment. https://www.cyclestreets.org/news/2021/07/25/mapping-ltns/ https://www.cyclestreets.org/news/2021/07/25/mapping-ltns/ is related.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/a-b-street/abstreet/issues/794#issuecomment-962680264, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADYURCWFLKE632N2IRMAEDUK3SRXANCNFSM5HBALDDA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[dabreegster]

https://a-b-street.github.io/docs/software/ltn/tech_details.html#neighborhood-selection has more details about how boundaries are defined today. What I'm hearing from nearly everyone is that picking the exact boundaries is pretty much impossible to do by heuristic, so I'm working on a UI to let people draw their own.