Open joeytalbot opened 1 year ago
Robinlovelace
commented
1 year ago Sorry to hear it's not working. Can you reproduce the minimal example here? https://github.com/dabreegster/odjitter/blob/main/r/R/jitter.R#L33-L43
If not it's probably an issue with your installation of odjitter. If so it's probably an issue with the input data. Good luck with it!
joeytalbot
commented
1 year ago The example works. There must be some issues with the data.
Robinlovelace
commented
1 year ago OK, happy to take a look if you share a reproducible example. Good luck with it!
joeytalbot
commented
1 year ago I've managed to fix that error somehow, but now I'm getting a different one!
> od_foot_jittered = odjitter::jitter(
+ od = lines_foot,
+ zones = northeast,
+ # zone_name_key = "geo_code",
+ subpoints = osm_drive,
+ disaggregation_threshold = disag_threshold,
+ disaggregation_key = "foot",
+ min_distance_meters = min_distance_meters
+ )
Scraped 1460 zones from C:\Users\geojta\AppData\Local\Temp\Rtmpawvs7M/zones.geojson
Scraped 901049 subpoints from C:\Users\geojta\AppData\Local\Temp\Rtmpawvs7M/subpoints.geojson
Scraped 901049 subpoints from C:\Users\geojta\AppData\Local\Temp\Rtmpawvs7M/subpoints.geojson
Disaggregating OD data
thread 'main' panicked at 'no entry found for key', C:\Users\geojta\.cargo\git\checkouts\odjitter-2ee039e64a6d7c8d\00ba4b0\src\lib.rs:196:55
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: Cannot open "C:\Users\geojta\AppData\Local\Temp\Rtmpawvs7M\od_jittered.geojson"; The source could be corrupt or not supported. See `st_drivers()` for a list of supported formats.
In addition: Warning messages:
1: In CPL_read_ogr(dsn, layer, query, as.character(options), quiet, :
GDAL Error 1: At line 2, character 1: Unterminated array
2: In CPL_read_ogr(dsn, layer, query, as.character(options), quiet, :
GDAL Error 4: Failed to read GeoJSON dataThere may be a problem with accessing "C:\Users\geojta\AppData\Local\Temp\Rtmpawvs7M\od_jittered.geojson", but this file does exist.
joeytalbot
commented
1 year ago Here's a reprex:
# Tyne and Wear census data -----------------------------------------------
library(pct)
#> Warning: package 'pct' was built under R version 4.1.3
library(tidyverse)
#> Warning: package 'ggplot2' was built under R version 4.1.3
#> Warning: package 'tibble' was built under R version 4.1.3
#> Warning: package 'readr' was built under R version 4.1.3
#> Warning: package 'purrr' was built under R version 4.1.3
#> Warning: package 'dplyr' was built under R version 4.1.3
#> Warning: package 'stringr' was built under R version 4.1.3
northeast = get_pct_zones("north-east")
northeast = northeast %>%
select(geo_code)
lines = get_pct(region = "north-east", purpose = "commute", geography = "msoa", layer = "l")
lines_tyneandwear = lines %>%
filter(lad_name1 == "Newcastle upon Tyne" | lad_name1 == "Sunderland" |
lad_name1 == "Gateshead" | lad_name1 == "North Tyneside" |
lad_name1 =="South Tyneside" | lad_name2 == "Newcastle upon Tyne" |
lad_name2 == "Sunderland" | lad_name2 == "Gateshead" |
lad_name2 == "North Tyneside" | lad_name2 =="South Tyneside"
)
lines_car = lines_tyneandwear %>%
filter(car_driver > 0) %>%
select(geo_code1, geo_code2, car_driver)
# OSM data ----------------------------------------------------------------
# Tags
et = c(
"maxspeed",
"oneway",
"lanes",
"ref",
"surface",
"segregated",
"state",
"network"
)
# Read-in road network data
osm_lines1 = osmextract::oe_get_network(
place = "Tyne and Wear",
mode = "driving",
extra_tags = et
)
#> The input place was matched with: Tyne and Wear
#> | | | 0% | | | 1% | |= | 1% | |= | 2% | |== | 2% | |== | 3% | |== | 4% | |=== | 4% | |=== | 5% | |==== | 5% | |==== | 6% | |===== | 6% | |===== | 7% | |===== | 8% | |====== | 8% | |====== | 9% | |======= | 9% | |======= | 10% | |======= | 11% | |======== | 11% | |======== | 12% | |========= | 12% | |========= | 13% | |========= | 14% | |========== | 14% | |========== | 15% | |=========== | 15% | |=========== | 16% | |============ | 16% | |============ | 17% | |============ | 18% | |============= | 18% | |============= | 19% | |============== | 19% | |============== | 20% | |============== | 21% | |=============== | 21% | |=============== | 22% | |================ | 22% | |================ | 23% | |================ | 24% | |================= | 24% | |================= | 25% | |================== | 25% | |================== | 26% | |=================== | 26% | |=================== | 27% | |=================== | 28% | |==================== | 28% | |==================== | 29% | |===================== | 29% | |===================== | 30% | |===================== | 31% | |====================== | 31% | |====================== | 32% | |======================= | 32% | |======================= | 33% | |======================= | 34% | |======================== | 34% | |======================== | 35% | |========================= | 35% | |========================= | 36% | |========================== | 36% | |========================== | 37% | |========================== | 38% | |=========================== | 38% | |=========================== | 39% | |============================ | 39% | |============================ | 40% | |============================ | 41% | |============================= | 41% | |============================= | 42% | |============================== | 42% | |============================== | 43% | |============================== | 44% | |=============================== | 44% | |=============================== | 45% | |================================ | 45% | |================================ | 46% | |================================= | 46% | |================================= | 47% | |================================= | 48% | |================================== | 48% | |================================== | 49% | |=================================== | 49% | |=================================== | 50% | |=================================== | 51% | |==================================== | 51% | |==================================== | 52% | |===================================== | 52% | |===================================== | 53% | |===================================== | 54% | |====================================== | 54% | |====================================== | 55% | |======================================= | 55% | |======================================= | 56% | |======================================== | 56% | |======================================== | 57% | |======================================== | 58% | |========================================= | 58% | |========================================= | 59% | |========================================== | 59% | |========================================== | 60% | |========================================== | 61% | |=========================================== | 61% | |=========================================== | 62% | |============================================ | 62% | |============================================ | 63% | |============================================ | 64% | |============================================= | 64% | |============================================= | 65% | |============================================== | 65% | |============================================== | 66% | |=============================================== | 66% | |=============================================== | 67% | |=============================================== | 68% | |================================================ | 68% | |================================================ | 69% | |================================================= | 69% | |================================================= | 70% | |================================================= | 71% | |================================================== | 71% | |================================================== | 72% | |=================================================== | 72% | |=================================================== | 73% | |=================================================== | 74% | |==================================================== | 74% | |==================================================== | 75% | |===================================================== | 75% | |===================================================== | 76% | |====================================================== | 76% | |====================================================== | 77% | |====================================================== | 78% | |======================================================= | 78% | |======================================================= | 79% | |======================================================== | 79% | |======================================================== | 80% | |======================================================== | 81% | |========================================================= | 81% | |========================================================= | 82% | |========================================================== | 82% | |========================================================== | 83% | |========================================================== | 84% | |=========================================================== | 84% | |=========================================================== | 85% | |============================================================ | 85% | |============================================================ | 86% | |============================================================= | 86% | |============================================================= | 87% | |============================================================= | 88% | |============================================================== | 88% | |============================================================== | 89% | |=============================================================== | 89% | |=============================================================== | 90% | |=============================================================== | 91% | |================================================================ | 91% | |================================================================ | 92% | |================================================================= | 92% | |================================================================= | 93% | |================================================================= | 94% | |================================================================== | 94% | |================================================================== | 95% | |=================================================================== | 95% | |=================================================================== | 96% | |==================================================================== | 96% | |==================================================================== | 97% | |==================================================================== | 98% | |===================================================================== | 98% | |===================================================================== | 99% | |======================================================================| 99% | |======================================================================| 100%
#> File downloaded!
#> Start with the vectortranslate operations on the input file!
#> 0...10...20...30...40...50...60...70...80...90...100 - done.
#> Finished the vectortranslate operations on the input file!
#> Reading layer `lines' from data source
#> `C:\Users\geojta\AppData\Local\Temp\Rtmp67TIH0\geofabrik_tyne-and-wear-latest.gpkg'
#> using driver `GPKG'
#> Simple feature collection with 54146 features and 19 fields
#> Geometry type: LINESTRING
#> Dimension: XY
#> Bounding box: xmin: -1.878213 ymin: 54.79363 xmax: -1.346884 ymax: 55.08401
#> Geodetic CRS: WGS 84
osm_lines2 = osmextract::oe_get_network(
place = "Northumberland",
mode = "driving",
extra_tags = et
)
#> The input place was matched with: Northumberland
#> | | | 0% | | | 1% | |= | 1% | |= | 2% | |== | 2% | |== | 3% | |== | 4% | |=== | 4% | |=== | 5% | |==== | 5% | |==== | 6% | |===== | 6% | |===== | 7% | |===== | 8% | |====== | 8% | |====== | 9% | |======= | 9% | |======= | 10% | |======= | 11% | |======== | 11% | |======== | 12% | |========= | 12% | |========= | 13% | |========= | 14% | |========== | 14% | |========== | 15% | |=========== | 15% | |=========== | 16% | |============ | 16% | |============ | 17% | |============ | 18% | |============= | 18% | |============= | 19% | |============== | 19% | |============== | 20% | |============== | 21% | |=============== | 21% | |=============== | 22% | |================ | 22% | |================ | 23% | |================ | 24% | |================= | 24% | |================= | 25% | |================== | 25% | |================== | 26% | |=================== | 26% | |=================== | 27% | |=================== | 28% | |==================== | 28% | |==================== | 29% | |===================== | 29% | |===================== | 30% | |===================== | 31% | |====================== | 31% | |====================== | 32% | |======================= | 32% | |======================= | 33% | |======================= | 34% | |======================== | 34% | |======================== | 35% | |========================= | 35% | |========================= | 36% | |========================== | 36% | |========================== | 37% | |========================== | 38% | |=========================== | 38% | |=========================== | 39% | |============================ | 39% | |============================ | 40% | |============================ | 41% | |============================= | 41% | |============================= | 42% | |============================== | 42% | |============================== | 43% | |============================== | 44% | |=============================== | 44% | |=============================== | 45% | |================================ | 45% | |================================ | 46% | |================================= | 46% | |================================= | 47% | |================================= | 48% | |================================== | 48% | |================================== | 49% | |=================================== | 49% | |=================================== | 50% | |==================================== | 51% | |==================================== | 52% | |===================================== | 52% | |===================================== | 53% | |===================================== | 54% | |====================================== | 54% | |====================================== | 55% | |======================================= | 55% | |======================================= | 56% | |======================================== | 56% | |======================================== | 57% | |======================================== | 58% | |========================================= | 58% | |========================================= | 59% | |========================================== | 59% | |========================================== | 60% | |========================================== | 61% | |=========================================== | 61% | |=========================================== | 62% | |============================================ | 62% | |============================================ | 63% | |============================================ | 64% | |============================================= | 64% | |============================================= | 65% | |============================================== | 65% | |============================================== | 66% | |=============================================== | 66% | |=============================================== | 67% | |=============================================== | 68% | |================================================ | 68% | |================================================ | 69% | |================================================= | 69% | |================================================= | 70% | |================================================= | 71% | |================================================== | 71% | |================================================== | 72% | |=================================================== | 72% | |=================================================== | 73% | |=================================================== | 74% | |==================================================== | 74% | |==================================================== | 75% | |===================================================== | 75% | |===================================================== | 76% | |====================================================== | 76% | |====================================================== | 77% | |====================================================== | 78% | |======================================================= | 78% | |======================================================= | 79% | |======================================================== | 79% | |======================================================== | 80% | |======================================================== | 81% | |========================================================= | 81% | |========================================================= | 82% | |========================================================== | 82% | |========================================================== | 83% | |========================================================== | 84% | |=========================================================== | 84% | |=========================================================== | 85% | |============================================================ | 85% | |============================================================ | 86% | |============================================================= | 86% | |============================================================= | 87% | |============================================================= | 88% | |============================================================== | 88% | |============================================================== | 89% | |=============================================================== | 89% | |=============================================================== | 90% | |=============================================================== | 91% | |================================================================ | 91% | |================================================================ | 92% | |================================================================= | 92% | |================================================================= | 93% | |================================================================= | 94% | |================================================================== | 94% | |================================================================== | 95% | |=================================================================== | 95% | |=================================================================== | 96% | |==================================================================== | 96% | |==================================================================== | 97% | |==================================================================== | 98% | |===================================================================== | 98% | |===================================================================== | 99% | |======================================================================| 99% | |======================================================================| 100%
#> File downloaded!
#> Start with the vectortranslate operations on the input file!
#> 0...10...20...30...40...50...60...70...80...90...100 - done.
#> Finished the vectortranslate operations on the input file!
#> Reading layer `lines' from data source
#> `C:\Users\geojta\AppData\Local\Temp\Rtmp67TIH0\geofabrik_northumberland-latest.gpkg'
#> using driver `GPKG'
#> Simple feature collection with 34877 features and 19 fields
#> Geometry type: LINESTRING
#> Dimension: XY
#> Bounding box: xmin: -2.735143 ymin: 54.60299 xmax: -1.459093 ymax: 55.82127
#> Geodetic CRS: WGS 84
osm_lines3 = osmextract::oe_get_network(
place = "Durham",
mode = "driving",
extra_tags = et
)
#> The input place was matched with: Durham
#> | | | 0% | | | 1% | |= | 1% | |= | 2% | |== | 2% | |== | 3% | |== | 4% | |=== | 4% | |=== | 5% | |==== | 5% | |==== | 6% | |===== | 6% | |===== | 7% | |===== | 8% | |====== | 8% | |====== | 9% | |======= | 9% | |======= | 10% | |======= | 11% | |======== | 11% | |======== | 12% | |========= | 12% | |========= | 13% | |========= | 14% | |========== | 14% | |========== | 15% | |=========== | 15% | |=========== | 16% | |============ | 16% | |============ | 17% | |============ | 18% | |============= | 18% | |============= | 19% | |============== | 19% | |============== | 20% | |============== | 21% | |=============== | 21% | |=============== | 22% | |================ | 22% | |================ | 23% | |================ | 24% | |================= | 24% | |================= | 25% | |================== | 25% | |================== | 26% | |=================== | 26% | |=================== | 27% | |=================== | 28% | |==================== | 28% | |==================== | 29% | |===================== | 29% | |===================== | 30% | |===================== | 31% | |====================== | 31% | |====================== | 32% | |======================= | 32% | |======================= | 33% | |======================= | 34% | |======================== | 34% | |======================== | 35% | |========================= | 35% | |========================= | 36% | |========================== | 36% | |========================== | 37% | |========================== | 38% | |=========================== | 38% | |=========================== | 39% | |============================ | 39% | |============================ | 40% | |============================ | 41% | |============================= | 41% | |============================= | 42% | |============================== | 42% | |============================== | 43% | |============================== | 44% | |=============================== | 44% | |=============================== | 45% | |================================ | 45% | |================================ | 46% | |================================= | 46% | |================================= | 47% | |================================= | 48% | |================================== | 48% | |================================== | 49% | |=================================== | 49% | |=================================== | 50% | |=================================== | 51% | |==================================== | 51% | |==================================== | 52% | |===================================== | 52% | |===================================== | 53% | |===================================== | 54% | |====================================== | 54% | |====================================== | 55% | |======================================= | 55% | |======================================= | 56% | |======================================== | 56% | |======================================== | 57% | |======================================== | 58% | |========================================= | 58% | |========================================= | 59% | |========================================== | 59% | |========================================== | 60% | |========================================== | 61% | |=========================================== | 61% | |=========================================== | 62% | |============================================ | 62% | |============================================ | 63% | |============================================ | 64% | |============================================= | 64% | |============================================= | 65% | |============================================== | 65% | |============================================== | 66% | |=============================================== | 66% | |=============================================== | 67% | |=============================================== | 68% | |================================================ | 68% | |================================================ | 69% | |================================================= | 69% | |================================================= | 70% | |================================================= | 71% | |================================================== | 71% | |================================================== | 72% | |=================================================== | 72% | |=================================================== | 73% | |=================================================== | 74% | |==================================================== | 74% | |==================================================== | 75% | |===================================================== | 75% | |===================================================== | 76% | |====================================================== | 76% | |====================================================== | 77% | |====================================================== | 78% | |======================================================= | 78% | |======================================================= | 79% | |======================================================== | 79% | |======================================================== | 80% | |======================================================== | 81% | |========================================================= | 81% | |========================================================= | 82% | |========================================================== | 82% | |========================================================== | 83% | |========================================================== | 84% | |=========================================================== | 84% | |=========================================================== | 85% | |============================================================ | 85% | |============================================================ | 86% | |============================================================= | 86% | |============================================================= | 87% | |============================================================= | 88% | |============================================================== | 88% | |============================================================== | 89% | |=============================================================== | 89% | |=============================================================== | 90% | |=============================================================== | 91% | |================================================================ | 91% | |================================================================ | 92% | |================================================================= | 92% | |================================================================= | 93% | |================================================================= | 94% | |================================================================== | 94% | |================================================================== | 95% | |=================================================================== | 95% | |=================================================================== | 96% | |==================================================================== | 97% | |==================================================================== | 98% | |===================================================================== | 98% | |===================================================================== | 99% | |======================================================================| 99% | |======================================================================| 100%
#> File downloaded!
#> Start with the vectortranslate operations on the input file!
#> 0...10...20...30...40...50...60...70...80...90...100 - done.
#> Finished the vectortranslate operations on the input file!
#> Reading layer `lines' from data source
#> `C:\Users\geojta\AppData\Local\Temp\Rtmp67TIH0\geofabrik_durham-latest.gpkg'
#> using driver `GPKG'
#> Simple feature collection with 62808 features and 19 fields
#> Geometry type: LINESTRING
#> Dimension: XY
#> Bounding box: xmin: -2.356457 ymin: 54.38049 xmax: -1.162522 ymax: 54.93337
#> Geodetic CRS: WGS 84
osm_lines = bind_rows(osm_lines1, osm_lines2, osm_lines3)
to_exclude = "services|bridleway|disused|emergency|escap|far|foot|path|pedestrian|rest|road|track"
osm_drive = osm_lines %>%
filter(!str_detect(string = highway, pattern = to_exclude))
# Jittered routes ---------------------------------------------------------
min_distance_meters = 500
disag_threshold = 50
od_car_jittered = odjitter::jitter(
od = lines_car,
zones = northeast,
zone_name_key = "geo_code",
subpoints = osm_drive,
disaggregation_threshold = disag_threshold,
disaggregation_key = "car_driver",
min_distance_meters = min_distance_meters
)
#> Warning in CPL_read_ogr(dsn, layer, query, as.character(options), quiet, : GDAL
#> Error 1: At line 2, character 1: Unterminated array
#> Warning in CPL_read_ogr(dsn, layer, query, as.character(options), quiet, : GDAL
#> Error 4: Failed to read GeoJSON data
#> Error: Cannot open "C:\Users\geojta\AppData\Local\Temp\Rtmp67TIH0\od_jittered.geojson"; The source could be corrupt or not supported. See `st_drivers()` for a list of supported formats.Created on 2022-12-12 with reprex v2.0.2
joeytalbot
commented
1 year ago This could be important:
thread 'main' panicked at 'no entry found for key', C:\Users\geojta\.cargo\git\checkouts\odjitter-2ee039e64a6d7c8d\00ba4b0\src\lib.rs:196:55It's working! There were some geo_codes that didn't match, I think they were from more distant locations.
joeytalbot
commented
1 year ago The geo_codes that don't match are all from Northumberland. There was probably a change in the coding system there at some point recently, and we are trying to compare different versions of these codes.
joeytalbot
commented
1 year ago I've got the Northumberland zones now
joeytalbot
commented
1 year ago I've created jittered routes, but there's a new error when I try to convert them into a route network:
> car_rnet = overline(
+ car_osrm,
+ # attrib = c("car_driver", "car_passenger", "motorbike", "taxi_other")
+ attrib = "car_driver"
+ )
2022-12-12 13:40:46 constructing segments
2022-12-12 13:40:52 building geometry
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=03s
2022-12-12 13:40:55 simplifying geometry
large data detected, using regionalisation, nrow = 161011
Error in FUN(X[[i]], ...) : subscript out of boundsI suggest trying to cast the lines to LINESTRING form first with st_cast(). Does that help? I suggest sharing the file and asking @mem48 for any ideas if not.
joeytalbot
commented
1 year ago Thanks @Robinlovelace . The geometry is already LINESTRING. You can see the data here https://github.com/ITSLeeds/trafficsim/releases/download/0.1/car_jittered_osrm.Rds I've asked Malcolm too so hopefully will find a solution.
Robinlovelace
commented
1 year ago No problem Joey. Have you tried running it on a small subset of the data, e.g.:
names(car_osrm)
car_rnet = overline(
car_osrm[1:9, ],
attrib = "car_driver"
)Good luck with it and keep us posted on updates.
joeytalbot
commented
1 year ago Using the first 9 rows only, the function works correctly:
car_rnet = overline(
car_osrm[1:9, ],
attrib = "car_driver"
)Good to know. I suggest trying different subsets of the data to try to identify the one that's failing. Good luck with it and sounds like progress!
joeytalbot
commented
1 year ago It seems to break when it tries to use regionalisation. Maybe we can avoid that entirely:
> car_rnet = overline(
+ car_osrm[1:3009, ],
+ attrib = "car_driver"
+ )
2022-12-12 16:24:07 constructing segments
2022-12-12 16:24:10 building geometry
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01s
2022-12-12 16:24:11 simplifying geometry
2022-12-12 16:24:11 aggregating flows
2022-12-12 16:24:14 rejoining segments into linestrings
> car_rnet = overline(
+ car_osrm[1:5009, ],
+ attrib = "car_driver"
+ )
2022-12-12 16:24:19 constructing segments
2022-12-12 16:24:23 building geometry
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=02s
2022-12-12 16:24:25 simplifying geometry
large data detected, using regionalisation, nrow = 115491
Error in FUN(X[[i]], ...) : subscript out of boundsYes, I think regionalisation should always be off. Maybe open (and try to fix) an issue in stplanr on that?
joeytalbot
commented
1 year ago Good idea, I've done so https://github.com/ropensci/stplanr/issues/510
I've extracted OSM road data for cycling and driving, but I haven't been able to use this yet for a jittered route network due to the following error: