Calculate 'in site' metrics

[Robinlovelace]

Split-out from https://github.com/cyipt/actdev-ui/issues/13#issuecomment-779914120 after communications with @joeytalbot. My thinking is that this should be fairly doable seeing as we have the site data. Possible method building on @si-the-pie's suggestion on quantifying rat-running in the context of LTNs:

• [x] Randomly sample points on the existing network in/around the site (e.g. within a 500m buffer of the site boundary)
• [x] Calculate walking, cycling driving routes between all OD pairs from the sample
• [x] Generate summary statistics of on-site walkability (which could just be car circuity divided by walking circuity with high values being good or vice versa) / cyclability (busyness + circuity stats) and save them as a file, e.g. called site-metrics.csv.

That's just the data side. More work would need to be done on the UI side to incorporate this. My sense is that it's a stretch (marking it as such) and possibly a Phase II issue but happy to revisit that if people want to give it a shot after lower hanging fruit of perhaps such as #90 have been fixed.

[joeytalbot]

Nice idea.

This is to address a lack of focus on the sites themselves. So we should use the site boundaries, not a 500m buffer.

The most likely problem with the layout is American style cul-de-sacs and winding roads. So we should simply measure circuity of walking routes. There's no need to complicate things by including the driving routes. Busyness is also fairly irrelevant. These will be residential roads in most cases.

[joeytalbot]

If we can calculate a simple metric of the mean circuity of walking routes within the sites, it would be easy to include this in the UI.

[Robinlovelace]

So we should use the site boundaries, not a 500m buffer.

That's an option but risks not showing issues with lack of simple missing links as highlighted in the map that triggered this by @mvl22 in https://github.com/cyipt/actdev-ui/issues/13#issuecomment-779914120. We shouldn't jump to conclusions, can see +s of taking some sample routing points outside the site boundary on the existing network (primarily to highlight the need for links to enable people to get out onto other roads - the opposite of what we see in Chapelford). What are the -s?

If we can calculate a simple metric of the mean circuity of walking routes within the sites, it would be easy to include this in the UI.

Happy to revisit this after more pressing issues are fixed - it's a stretch but doable!

[joeytalbot]

That's an option but risks not showing issues with lack of simple missing links as highlighted in the map that triggered this by @mvl22 in cyipt/actdev-ui#13 (comment). We shouldn't jump to conclusions, can see +s of taking some sample routing points outside the site boundary on the existing network (primarily to highlight the need for links to enable people to get out onto other roads - the opposite of what we see in Chapelford). What are the -s?

I disagree. The point of this is to look purely at the internal site layout. Stating that clearly will allow us to produce a metric that people can find useful.

The presence or lack of links across the site boundary is addressed by a different metric - "number of locations at which routes cross the site boundary."

See https://github.com/cyipt/actdev-ui/issues/13#issuecomment-779917732

[Robinlovelace]

I'm open minded about the best way forward and can see pros and cons both ways. Cannot see big negatives of going a bit outside site boundary but agree there are possible +s of constraining points to just inside site boundaries. Another possible con of creating a new metric: there are already quite a few different metrics and simplicity is good.

Not a priority in any case, one to sleep on, ask others about and revisit I suggest. Great to have stretch ideas hashed out up-front and no problem if we don't hit this - can always re-lable 'stretch' issues Phase II ones.

[mvl22]

If we can calculate a simple metric of the mean circuity of walking routes within the sites, it would be easy to include this in the UI.

That would be a clever way of determining whether a site is an old-fashioned DB32-style layout or a more modern principles Manual For Street layout, and indeed make a good 'key metric' for the site in internal terms.

[Robinlovelace]

Next step: write function that takes number of points, buffer size, and returns info on circuity for different modes.

• circuity_walk
• circuity_cycle
• circuity_drive

Will only do it for sites that have been built.

Stretch: % shared.

[Robinlovelace]

Has it built yet (is it OSM).

[joeytalbot]

I've added a column to all-sites.geojson on whether the site has been built yet.

[Robinlovelace]

Great to see, thanks @joeytalbot !

[Robinlovelace]

I'm working on this.

[Robinlovelace]

See below the approach. Heads-up @joeytalbot

``` r # Aim: calculate in-site metrics # Question: should we base this on OSM data or routing data? # It's quicker and less resource intensive to calculate # these metrics using a simple routing approach, reducing # the computational requirements associated with bulk # import and processing of OSM, although that 2nd # approach allows more detailed info to be extracted # Approach1: quicker and easier ------------------------------------------- # get starting point from infographics.R # file.edit("code/infographics.R") library(tidyverse) setwd("~/cyipt/actdev/") if(!exists("site_name")) site_name = "great-kneighton" sites = sf::read_sf("data-small/all-sites.geojson") site = sites[sites$site_name == site_name, ] path = file.path("data-small", site_name) # input data: we should probably have naming conventions for these list.files(path) #> [1] "all-census-od.csv" "buildings_od.geojson" #> [3] "dartboard-1-3-6km.geojson" "dartboard.geojson" #> [5] "desire_lines_disag.geojson" "desire-lines-few.geojson" #> [7] "desire-lines-many.geojson" "gg_busyness_dartboard.png" #> [9] "gg_distance_busyness.png" "jts-lsoas.geojson" #> [11] "large-study-area.geojson" "mode-split.csv" #> [13] "mode-split.png" "procgen_houses.json.gz" #> [15] "rnet-balanced.geojson" "rnet-fast.geojson" #> [17] "rnet-quiet.geojson" "rnet-walk.geojson" #> [19] "routes_balanced.Rds" "routes_fast.Rds" #> [21] "routes_quiet.Rds" "routes_walk.Rds" #> [23] "routes-balanced.geojson" "routes-fast.geojson" #> [25] "routes-quiet.geojson" "routes-walk.geojson" #> [27] "scenario-base.json" "scenario-godutch.json" #> [29] "site.geojson" "small-study-area.geojson" site_area = sf::read_sf(file.path(path, "site.geojson")) desire_lines = sf::read_sf(file.path(path, "desire-lines-few.geojson")) routes_fast = sf::read_sf(file.path(path, "routes-fast.geojson")) routes_quiet = sf::read_sf(file.path(path, "routes-quiet.geojson")) routes_walk = readRDS(file.path(path, "routes_walk.Rds")) mapview::mapview(site_area) ```  ``` r # sample points inside the area for route analysis set.seed(2021) # reproducible results n_routes = 20 site_points_origin = sf::st_sample(site_area, n_routes) #> Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 7.0.0 #> although coordinates are longitude/latitude, st_intersects assumes that they are planar #> although coordinates are longitude/latitude, st_intersects assumes that they are planar #> although coordinates are longitude/latitude, st_intersects assumes that they are planar #> although coordinates are longitude/latitude, st_intersects assumes that they are planar site_points_destination = sf::st_sample(site_area, n_routes) #> although coordinates are longitude/latitude, st_intersects assumes that they are planar #> although coordinates are longitude/latitude, st_intersects assumes that they are planar #> although coordinates are longitude/latitude, st_intersects assumes that they are planar #> although coordinates are longitude/latitude, st_intersects assumes that they are planar site_odc = cbind( sf::st_coordinates(site_points_origin), sf::st_coordinates(site_points_destination) ) site_desire_lines = od::odc_to_sf(site_odc) mapview::mapview(site_desire_lines) #> Warning in cbind(`Feature ID` = fid, mat): number of rows of result is not a #> multiple of vector length (arg 1) ```  ``` r # Approach 2: use data from osmextract ------------------------------------ ``` ^{Created on 2021-02-22 by the [reprex package](https://reprex.tidyverse.org) (v1.0.0)}

[Robinlovelace]

In-site walkability:

> site_desire_lines$length = geo_length(site_desire_lines)
> site_walking_circuity = sum(site_routes_walk$distance) / sum(site_desire_lines$length)
> site_walking_circuity = sum(site_routes_walk$distance) / sum(site_desire_lines$length)
> site_walking_circuity
[1] 1.313365

[Robinlovelace]

Results for Great Kneighton look good:

> site_drive_circuity = sum(site_routes_drive$distance) / sum(site_desire_lines$length)
> site_drive_circuity
[1] 2.430559

[Robinlovelace]

Not bad for cycling:

> site_cycle_circuity = sum(site_routes_cycle$distances) / sum(site_desire_lines$length)
> site_cycle_circuity

[Robinlovelace]

Heads-up @joeytalbot, @mvl22 and @Siequnu here's a site level infographic. This highlights the wider issue: we need project level colours for each mode of transport, something that would have been good to have earlier, e.g. in #48, but I've gone for:

• [ ] Walk: blue
• [ ] Cycle: green
• [ ] Car: red

Can we do better than this? Loving how it's looking in any case, sure we can improve on this:

[Robinlovelace]

Next step on this: move into main code folder and add new columns to all-sites.geojson. @joeytalbot is up for giving it a go. I'm happy to have a bash at it on Weds evening if not sorted by then.

[Robinlovelace]

Plan is for @joeytalbot to work on this today and put files like site-routes-walk.geojson, site-routes-cycle.geojson and site-routes-drive.geojson in folders associated with sites that have actually been built. One question for @mvl22 on the UI side, would it be possible/desirable for the UI to be able to show these on-site routes?

[Robinlovelace]

I think the infographic above could also be improved, leading to files like on-site-infographic.png in the site folders.

[joeytalbot]

At Ebbsfleet the in-site circuity and route distances shown are longer for cycling than for driving, even though the reverse seems true from looking at the maps. I'm investigating.

[Robinlovelace]

Cheers for the update, any figures/outputs welcome. Good you're taking a look at the data and doing sanity checks rather than just accepting the data at face value, wonder what is going on.

[joeytalbot]

It looks like a weird bug with the first route segment appearing multiple times.

[joeytalbot]

[Robinlovelace]

I think that's because cyclestreets::journey() generated data per segment while stplanr::route_osrm() generates data per route.

[Robinlovelace]

Quick fix: use route_osrm() for all modes (walking and driving currently, I suggest setting osrm.profile = "bike" as documented here: https://docs.ropensci.org/stplanr/reference/route_osrm.html

Or you could adapt the code to accomodate the segment level outputs from the journey() function. The quick fix is probably quicker and more consistent so I would go with that.

[joeytalbot]

Quick fix: use route_osrm() for all modes (walking and driving currently, I suggest setting osrm.profile = "bike" as documented here: https://docs.ropensci.org/stplanr/reference/route_osrm.html

I've done this.

[joeytalbot]

Tmap gives this for rnet maps

[joeytalbot]

Ok I've got it working

[Siequnu]

We can show any geojson we like, and I’ve built a scrollable site-data menu so we can potentially expand the current set.

Once we have a working site-level zoom site I’d say we can spend a few days end of next week optimising exactly which layers get shown.

would it be possible/desirable for the UI to be able to show these on-site routes?

[joeytalbot]

The comparison between in-site circuity of walking, cycling and driving routes isn't giving reliable results because the routes don't always spawn in the same place. Driving routes always spawn on roads, while walking and cycling routes can spawn on paths, which sometimes means they end up being much longer than the driving routes.

[joeytalbot]

The cycle route highlighted here should be identical to the driving route highlighted in the previous map, but it's much longer because it spawns on a path away from the housing estate.

[Robinlovelace]

Interesting and well spotted. Idea: use lwgeom::st_start_point() and lwgeom::st_end_point() to update the start and end points after the driving routes have been calculated. What do you reckon @joeytalbot ?

Very glad this was discovered early as it would have been a struggle to explain some results based on these flawed routes (also wonder what % it's affecting).

[joeytalbot]

Thanks for the tip. I don't think it's affecting a very high % of routes, but if we were to take a buffer around the site area it would affect many more routes, with higher chance of ending up on a small path.

[Robinlovelace]

Great. So plan is to go with something like this?

https://github.com/cyipt/actdev/blob/e19f56c6b241f2f62917b3a9ab0bab40208205cd/code/tests/in-site-metrics.R#L48-L68

In fact I've made a PR on this: #101

[joeytalbot]

Thanks, but I've already fixed it in here https://github.com/cyipt/actdev/commit/c34bbba8a9db6e7156fea75925d1ca6def2cb458

[Robinlovelace]

Great work, look forward to seeing the data + relative circuity values generated but this updated script.

[mvl22]

One question for @mvl22 on the UI side, would it be possible/desirable for the UI to be able to show these on-site routes?

Yes.

[joeytalbot]

I have now added all the data, including rnet geojsons, png graphics and new columns in `all-sites.geojson'

[Robinlovelace]

Just taking a look for allerton bywater, looking good to me!

[Robinlovelace]

The rnet values give a good indication of centrality on the in-site transport network:

[Robinlovelace]

Source: https://github.com/cyipt/actdev/blob/main/data-small/allerton-bywater/in-site-cycle-rnet.geojson

[Robinlovelace]

Walking routes look a bit more direct, as expected.

Source: https://github.com/cyipt/actdev/blob/main/data-small/allerton-bywater/in-site-walk-rnet.geojson

[Robinlovelace]

And great to see the updated info in all-sites.geojson. Heads-up @mvl22 and @Siequnu these new columns are ready to go and should be in the site.geojson file soon. Even more site metrics, closing this ahead of schedule :tada:

Any further issues on this you see @mvl22 let us know but I think there's plenty to visualise now and that this will greatly improve the prototype app and hopefully the final app in Phase II.

[Robinlovelace]