bristol_od - Githubissues

edzer commented 5 years ago

I'm seeing this error:

> example(bristol_od, run.dontrun=TRUE)
...
brstl_> bristol_region = getbb("Bristol", format_out = "sf_polygon") %>%
brstl_+         st_set_crs(4326) %>%
brstl_+         st_sf(data.frame(Name = "Bristol (OSM)"), geometry = region$geometry)
Error in st_sf(., data.frame(Name = "Bristol (OSM)"), geometry = region$geometry) : 
  object 'region' not found

``` > sessionInfo() R version 3.5.1 (2018-07-02) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 18.04.1 LTS Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1 locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] ukboundaries_0.2.0 forcats_0.3.0 stringr_1.3.1 [4] dplyr_0.7.99.9000 purrr_0.2.5.9000 readr_1.2.1 [7] tidyr_0.8.1 tibble_2.0.0 ggplot2_3.1.0 [10] tidyverse_1.2.1 sf_0.7-2 osmdata_0.0.8.099 [13] spDataLarge_0.2.7.3 loaded via a namespace (and not attached): [1] Rcpp_1.0.0 lubridate_1.7.4 lattice_0.20-35 prettyunits_1.0.2 [5] class_7.3-14 ps_1.1.0 clisymbols_1.2.0 assertthat_0.2.0 [9] rprojroot_1.3-2 digest_0.6.18 cellranger_1.1.0 R6_2.3.0 [13] plyr_1.8.4 backports_1.1.2 e1071_1.7-0 httr_1.3.1 [17] pillar_1.3.0.9001 rlang_0.3.0.9000 readxl_1.1.0 lazyeval_0.2.1 [21] curl_3.2 rstudioapi_0.7 callr_3.0.0 desc_1.2.0 [25] devtools_2.0.1 munsell_0.5.0 broom_0.5.0 modelr_0.1.2 [29] compiler_3.5.1 pkgconfig_2.0.2 base64enc_0.1-3 pkgbuild_1.0.2 [33] tcltk_3.5.1 tidyselect_0.2.5 crayon_1.3.4 withr_2.1.2 [37] grid_3.5.1 nlme_3.1-137 spData_0.2.9.4 jsonlite_1.5 [41] gtable_0.2.0 DBI_1.0.0 magrittr_1.5 units_0.6-1 [45] scales_1.0.0 cli_1.0.1 stringi_1.2.4 fs_1.2.6 [49] remotes_2.0.2 sp_1.3-1 testthat_2.0.0 xml2_1.2.0 [53] tools_3.5.1 glue_1.3.0 hms_0.4.2 processx_3.2.0 [57] pkgload_1.0.2 colorspace_1.4-0 sessioninfo_1.0.0 classInt_0.2-3 [61] rvest_0.3.2 memoise_1.1.0 haven_1.1.2 usethis_1.4.0.9000 ```

Robinlovelace commented 5 years ago

Fair point. The script I used to generate that dataset is not reproducible and the @examples just provide an indication. Will aim to tidy-up asap. Thanks for reporting @edzer - didn't know about the run.dontrun arg!

Robinlovelace commented 5 years ago

I think that fixes the region part. @edzer if you find any other non-reproducible bits in there let me know. Know the bristol_od dataset relies on manually downloading data that I couldn't find a way to download from within R.

edzer commented 5 years ago

Thanks -- I see. I think in that case the suggestion is to put this as a demo script in the demo directory, with sufficient warnings in comment that it is not actually meant to be runnable.

In case you want to make it runnable, you can copy the files you downloaded to a directory in /inst (if you are allowed), and have the example section read it from there. That would really help future users doing something similar.

Robinlovelace commented 5 years ago

Heads-up: I've fixed the reproducibility issue reported by uploading a small version of the file that can be accessed online from R. However I've discovered another problem: od2line() is no longer working. I suspect this may be linked to https://github.com/ropensci/stplanr/issues/271 ( @edzer is that linked to an email you sent a while back about this?) . Reprex below showing that the region and od dataset download code is now working:

devtools::install_github("ropensci/osmdata")
#> Skipping install of 'osmdata' from a github remote, the SHA1 (931dea30) has not changed since last install.
#>   Use `force = TRUE` to force installation
devtools::install_github("robinlovelace/ukboundaries")
#> Skipping install of 'ukboundaries' from a github remote, the SHA1 (1d162f0d) has not changed since last install.
#>   Use `force = TRUE` to force installation
library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. http://www.openstreetmap.org/copyright
library(sf)
#> Linking to GEOS 3.6.2, GDAL 2.2.3, PROJ 4.9.3
library(tidyverse)
library(ukboundaries)
#> Using default data cache directory ~/.ukboundaries/cache 
#> Use cache_dir() to change it.
#> Contains National Statistics data © Crown copyright and database right2018
#> Contains OS data © Crown copyright and database right, 2018
#> See https://www.ons.gov.uk/methodology/geography/licences
library(stplanr)
bristol_region = getbb("Bristol", format_out = "sf_polygon") %>%
        st_set_crs(4326) %>%
        st_sf(data.frame(Name = "Bristol (OSM)"), geometry = .$geometry)
mapview::mapview(bristol_region)
usethis::use_data(bristol_region, overwrite = TRUE)
#> Warning in path_file(base_path): restarting interrupted promise evaluation
#> Error: Path '/tmp/Rtmpdrg5fU/reprex3f7623a9b41' does not appear to be inside a project or package.
bristol_ttwa = ttwa_simple %>%
        filter(ttwa11nm == "Bristol") %>%
        select(Name = ttwa11nm)
bristol_ttwa$Name = "Bristol (TTWA)"
mapview::mapview(bristol_ttwa)
usethis::use_data(bristol_ttwa, overwrite = TRUE)
#> Warning in path_file(base_path): restarting interrupted promise evaluation
#> Error: Path '/tmp/Rtmpdrg5fU/reprex3f7623a9b41' does not appear to be inside a project or package.
bristol_cents = st_centroid(msoa2011_vsimple)[bristol_ttwa, ]
#> Warning in st_centroid.sf(msoa2011_vsimple): st_centroid assumes attributes
#> are constant over geometries of x
#> Warning in st_centroid.sfc(st_geometry(x), of_largest_polygon =
#> of_largest_polygon): st_centroid does not give correct centroids for
#> longitude/latitude data
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
plot(bristol_cents$geometry)
bristol_zones = msoa2011_vsimple[msoa2011_vsimple$msoa11cd %in% bristol_cents$msoa11cd, ] %>%
        select(geo_code = msoa11cd, name = msoa11nm) %>%
        mutate_at(1:2, as.character)
plot(bristol_zones$geometry, add = TRUE)

# Add travel data to the zones
# commented out - manual download from: http://wicid.ukdataservice.ac.uk/
# unzip("~/Downloads/wu03ew_v2.zip")
# od_all = read_csv("wu03ew_v2.csv")
# od_all_small = od_all %>%
#   filter(`Area of residence` %in% bristol_zones$geo_code &
#     `Area of workplace` %in% bristol_zones$geo_code,
#       `All categories: Method of travel to work` > 19)
# readr::write_csv(od_all_small, "od_all_small.csv")
# piggyback::pb_upload("od_all_small.csv")
# file.remove("wu03ew_v2.csv", "julyukrelease_tcm77-369384.xls")
od_all = read_csv("https://github.com/Nowosad/spDataLarge/releases/download/0.2.7.3/od_all_small.csv")
#> Parsed with column specification:
#> cols(
#>   `Area of residence` = col_character(),
#>   `Area of workplace` = col_character(),
#>   `All categories: Method of travel to work` = col_double(),
#>   `Work mainly at or from home` = col_double(),
#>   `Underground, metro, light rail, tram` = col_double(),
#>   Train = col_double(),
#>   `Bus, minibus or coach` = col_double(),
#>   Taxi = col_double(),
#>   `Motorcycle, scooter or moped` = col_double(),
#>   `Driving a car or van` = col_double(),
#>   `Passenger in a car or van` = col_double(),
#>   Bicycle = col_double(),
#>   `On foot` = col_double(),
#>   `Other method of travel to work` = col_double()
#> )
bristol_od = od_all %>%
       select(o = `Area of residence`, d = `Area of workplace`,
              all = `All categories: Method of travel to work`,
              bicycle = Bicycle, foot = `On foot`,
              car_driver = `Driving a car or van`, train = Train) %>%
       filter(o %in% bristol_zones$geo_code & d %in% bristol_zones$geo_code, all > 19)
summary(bristol_zones$geo_code %in% bristol_od$d)
#>    Mode    TRUE 
#> logical     102
summary(bristol_zones$geo_code %in% bristol_od$o)
#>    Mode    TRUE 
#> logical     102
usethis::use_data(bristol_zones, overwrite = TRUE)
#> Warning in path_file(base_path): restarting interrupted promise evaluation
#> Error: Path '/tmp/Rtmpdrg5fU/reprex3f7623a9b41' does not appear to be inside a project or package.
usethis::use_data(bristol_od, overwrite = TRUE)
#> Warning in path_file(base_path): restarting interrupted promise evaluation
#> Error: Path '/tmp/Rtmpdrg5fU/reprex3f7623a9b41' does not appear to be inside a project or package.
od_intra = filter(bristol_od, o == d)
od_inter = filter(bristol_od, o != d)
desire_lines = od2line(od_inter, zones) # failing here
#> Warning in od2line.Spatial(od_inter, zones): No match for line 1
#> Error in .local(obj, ...): cannot derive coordinates from non-numeric matrix
desire_lines$distance = as.numeric(st_length(desire_lines))
#> Error in st_geometry(x): object 'desire_lines' not found
desire_carshort = dplyr::filter(desire_lines, car_driver > 300 & distance < 5000)
#> Error in dplyr::filter(desire_lines, car_driver > 300 & distance < 5000): object 'desire_lines' not found
route_carshort = stplanr::line2route(desire_carshort, route_fun = route_osrm)
#> Error in is(l, "sf"): object 'desire_carshort' not found
bb = st_bbox(bristol_ttwa)
ways_road = opq(bbox = bb) %>%
        add_osm_feature(key = "highway",
                        value = "motorway|cycle|primary|secondary",
                        value_exact = FALSE) %>%
        osmdata_sf()
ways_rail = opq(bbox = bb) %>%
        add_osm_feature(key = "railway", value = "rail") %>%
        osmdata_sf()
res = c(ways_road, ways_rail)
summary(res)
#>                   Length Class  Mode     
#> bbox                1    -none- character
#> overpass_call       1    -none- character
#> meta                3    -none- list     
#> osm_points        102    sf     list     
#> osm_lines         185    sf     list     
#> osm_polygons      165    sf     list     
#> osm_multilines      0    -none- NULL     
#> osm_multipolygons   0    -none- NULL
bristol_stations = res$osm_points %>%
        filter(railway == "station" | name == "Bristol Temple Meads")
# most important vars:
map_int(bristol_stations, ~ sum(is.na(.))) %>%
        sort() %>%
        head()
#>          osm_id            name        geometry naptan.AtcoCode 
#>               0               0               0               1 
#>         railway        wikidata 
#>               1              11
bristol_stations = bristol_stations %>% select(name)
usethis::use_data(bristol_stations)
#> Warning in path_file(base_path): restarting interrupted promise evaluation
#> Error: Path '/tmp/Rtmpdrg5fU/reprex3f7623a9b41' does not appear to be inside a project or package.
ways = st_intersection(res$osm_lines, bristol_ttwa)
#> although coordinates are longitude/latitude, st_intersection assumes that they are planar
#> Warning: attribute variables are assumed to be spatially constant
#> throughout all geometries
ways$highway = as.character(ways$highway)
ways$highway[ways$railway == "rail"] = "rail"
ways$highway = gsub("_link", "", x = ways$highway) %>%
        gsub("motorway|primary|secondary", "road", x = .) %>%
        as.factor()
ways = ways %>%
        select(highway, maxspeed, ref)
summary(st_geometry_type(ways))
#>           GEOMETRY              POINT         LINESTRING 
#>                  0                  0               4903 
#>            POLYGON         MULTIPOINT    MULTILINESTRING 
#>                  0                  0                 12 
#>       MULTIPOLYGON GEOMETRYCOLLECTION     CIRCULARSTRING 
#>                  0                  0                  0 
#>      COMPOUNDCURVE       CURVEPOLYGON         MULTICURVE 
#>                  0                  0                  0 
#>       MULTISURFACE              CURVE            SURFACE 
#>                  0                  0                  0 
#>  POLYHEDRALSURFACE                TIN           TRIANGLE 
#>                  0                  0                  0
# convert to linestring
bristol_ways = st_cast(ways, "LINESTRING")
#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only
#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only
summary(st_geometry(bristol_ways))
#>    LINESTRING     epsg:4326 +proj=long... 
#>          4915             0             0
usethis::use_data(bristol_ways)
#> Warning in path_file(base_path): restarting interrupted promise evaluation
#> Error: Path '/tmp/Rtmpdrg5fU/reprex3f7623a9b41' does not appear to be inside a project or package.

^{Created on 2018-11-30 by the reprex package (v0.2.1)}

Robinlovelace commented 5 years ago

The latest version of the code is fully reproducible. I think this can be closed ( please review @Nowosad - I think the updated code and data are ready to merge):

devtools::install_github("ropensci/osmdata")
#> Skipping install of 'osmdata' from a github remote, the SHA1 (931dea30) has not changed since last install.
#>   Use `force = TRUE` to force installation
devtools::install_github("robinlovelace/ukboundaries")
#> Skipping install of 'ukboundaries' from a github remote, the SHA1 (1d162f0d) has not changed since last install.
#>   Use `force = TRUE` to force installation
library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. http://www.openstreetmap.org/copyright
library(sf)
#> Linking to GEOS 3.6.2, GDAL 2.2.3, PROJ 4.9.3
library(tidyverse)
library(ukboundaries)
#> Using default data cache directory ~/.ukboundaries/cache 
#> Use cache_dir() to change it.
#> Contains National Statistics data © Crown copyright and database right2018
#> Contains OS data © Crown copyright and database right, 2018
#> See https://www.ons.gov.uk/methodology/geography/licences
library(stplanr)
bristol_region = getbb("Bristol", format_out = "sf_polygon") %>%
 st_set_crs(4326) %>%
 st_sf(data.frame(Name = "Bristol (OSM)"), geometry = .$geometry)
mapview::mapview(bristol_region)
# usethisis::use_data(bristol_region, overwrite = TRUE)
bristol_ttwa = ttwa_simple %>%
 filter(ttwa11nm == "Bristol") %>%
 select(Name = ttwa11nm)
bristol_ttwa$Name = "Bristol (TTWA)"
mapview::mapview(bristol_ttwa)
# usethisis::use_data(bristol_ttwa, overwrite = TRUE)
bristol_cents = st_centroid(msoa2011_vsimple)[bristol_ttwa, ]
#> Warning in st_centroid.sf(msoa2011_vsimple): st_centroid assumes attributes
#> are constant over geometries of x
#> Warning in st_centroid.sfc(st_geometry(x), of_largest_polygon =
#> of_largest_polygon): st_centroid does not give correct centroids for
#> longitude/latitude data
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
plot(bristol_cents$geometry)
bristol_zones = msoa2011_vsimple[msoa2011_vsimple$msoa11cd %in% bristol_cents$msoa11cd, ] %>%
 select(geo_code = msoa11cd, name = msoa11nm) %>%
 mutate_at(1:2, as.character)
plot(bristol_zones$geometry, add = TRUE)

# Add travel data to the zones
# commented out - manual download from: http://wicid.ukdataservice.ac.uk/
# unzip("~/Downloads/wu03ew_v2.zip")
# od_all = read_csv("wu03ew_v2.csv")
# od_all_small = od_all %>%
#   filter(`Area of residence` %in% bristol_zones$geo_code &
#     `Area of workplace` %in% bristol_zones$geo_code,
#       `All categories: Method of travel to work` > 19)
# readr::write_csv(od_all_small, "od_all_small.csv")
# piggyback::pb_upload("od_all_small.csv")
# file.remove("wu03ew_v2.csv", "julyukrelease_tcm77-369384.xls")
od_all = read_csv("https://github.com/Nowosad/spDataLarge/releases/download/0.2.7.3/od_all_small.csv")
#> Parsed with column specification:
#> cols(
#>   `Area of residence` = col_character(),
#>   `Area of workplace` = col_character(),
#>   `All categories: Method of travel to work` = col_double(),
#>   `Work mainly at or from home` = col_double(),
#>   `Underground, metro, light rail, tram` = col_double(),
#>   Train = col_double(),
#>   `Bus, minibus or coach` = col_double(),
#>   Taxi = col_double(),
#>   `Motorcycle, scooter or moped` = col_double(),
#>   `Driving a car or van` = col_double(),
#>   `Passenger in a car or van` = col_double(),
#>   Bicycle = col_double(),
#>   `On foot` = col_double(),
#>   `Other method of travel to work` = col_double()
#> )
bristol_od = od_all %>%
select(o = `Area of residence`, d = `Area of workplace`,
       all = `All categories: Method of travel to work`,
       bicycle = Bicycle, foot = `On foot`,
       car_driver = `Driving a car or van`, train = Train) %>%
filter(o %in% bristol_zones$geo_code & d %in% bristol_zones$geo_code, all > 19)
summary(bristol_zones$geo_code %in% bristol_od$d)
#>    Mode    TRUE 
#> logical     102
summary(bristol_zones$geo_code %in% bristol_od$o)
#>    Mode    TRUE 
#> logical     102
# usethisis::use_data(bristol_zones, overwrite = TRUE)
# usethisis::use_data(bristol_od, overwrite = TRUE)
od_intra = filter(bristol_od, o == d)
od_inter = filter(bristol_od, o != d)
desire_lines = od2line(od_inter, bristol_zones)
#> Warning in st_centroid.sf(zones): st_centroid assumes attributes are
#> constant over geometries of x

#> Warning in st_centroid.sf(zones): st_centroid does not give correct
#> centroids for longitude/latitude data
desire_lines$distance = as.numeric(st_length(desire_lines))
desire_carshort = dplyr::filter(desire_lines, car_driver > 300 & distance < 5000)
route_carshort = stplanr::line2route(desire_carshort, route_fun = route_osrm)
#> 17 % out of 6 distances calculated
#> 33 % out of 6 distances calculated
#> 50 % out of 6 distances calculated
#> 67 % out of 6 distances calculated
#> 83 % out of 6 distances calculated
#> 100 % out of 6 distances calculated
route_carshort = st_as_sf(route_carshort)
# usethisis::use_data(route_carshort, overwrite = TRUE)
bb = st_bbox(bristol_ttwa)
ways_road = opq(bbox = bb) %>%
 add_osm_feature(key = "highway",
                 value = "motorway|cycle|primary|secondary",
                 value_exact = FALSE) %>%
 osmdata_sf()
ways_rail = opq(bbox = bb) %>%
 add_osm_feature(key = "railway", value = "rail") %>%
 osmdata_sf()
res = c(ways_road, ways_rail)
summary(res)
#>                   Length Class  Mode     
#> bbox                1    -none- character
#> overpass_call       1    -none- character
#> meta                3    -none- list     
#> osm_points        102    sf     list     
#> osm_lines         184    sf     list     
#> osm_polygons      165    sf     list     
#> osm_multilines      0    -none- NULL     
#> osm_multipolygons   0    -none- NULL
bristol_stations = res$osm_points %>%
 filter(railway == "station" | name == "Bristol Temple Meads")
# most important vars:
map_int(bristol_stations, ~ sum(is.na(.))) %>%
 sort() %>%
 head()
#>          osm_id            name        geometry naptan.AtcoCode 
#>               0               0               0               1 
#>         railway        wikidata 
#>               1              11
bristol_stations = bristol_stations %>% select(name)
# usethisis::use_data(bristol_stations, overwrite = TRUE)
ways = st_intersection(res$osm_lines, bristol_ttwa)
#> although coordinates are longitude/latitude, st_intersection assumes that they are planar
#> Warning: attribute variables are assumed to be spatially constant
#> throughout all geometries
ways$highway = as.character(ways$highway)
ways$highway[ways$railway == "rail"] = "rail"
ways$highway = gsub("_link", "", x = ways$highway) %>%
 gsub("motorway|primary|secondary", "road", x = .) %>%
 as.factor()
ways = ways %>%
 select(highway, maxspeed, ref)
summary(st_geometry_type(ways))
#>           GEOMETRY              POINT         LINESTRING 
#>                  0                  0               4903 
#>            POLYGON         MULTIPOINT    MULTILINESTRING 
#>                  0                  0                 12 
#>       MULTIPOLYGON GEOMETRYCOLLECTION     CIRCULARSTRING 
#>                  0                  0                  0 
#>      COMPOUNDCURVE       CURVEPOLYGON         MULTICURVE 
#>                  0                  0                  0 
#>       MULTISURFACE              CURVE            SURFACE 
#>                  0                  0                  0 
#>  POLYHEDRALSURFACE                TIN           TRIANGLE 
#>                  0                  0                  0
# convert to linestring
bristol_ways = st_cast(ways, "LINESTRING")
#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only
#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only
summary(st_geometry(bristol_ways))
#>    LINESTRING     epsg:4326 +proj=long... 
#>          4915             0             0
# usethisis::use_data(bristol_ways, overwrite = TRUE)

^{Created on 2018-12-01 by the reprex package (v0.2.1)}

edzer commented 5 years ago

For me, it stops at

brstl_> usethis::use_data(bristol_region, overwrite = TRUE)
Error: Path '.' does not appear to be inside a project or package.
In addition: Warning message:
In path_file(base_path) : restarting interrupted promise evaluation

Robinlovelace commented 5 years ago

Hi @edzer are you on the right branch?

The fixed version is on the https://github.com/Nowosad/spDataLarge/tree/bristol_od-fix branch.

Robinlovelace commented 5 years ago

And generates the following output on my computer:

devtools::install_github("nowosad/spDataLarge", ref = "bristol_od-fix")
#> Skipping install of 'spDataLarge' from a github remote, the SHA1 (1195a4b6) has not changed since last install.
#>   Use `force = TRUE` to force installation
library(spDataLarge)
example(bristol_od, run.dontrun=TRUE)
#> 
#> brstl_> devtools::install_github("ropensci/osmdata")
#> Skipping install of 'osmdata' from a github remote, the SHA1 (931dea30) has not changed since last install.
#>   Use `force = TRUE` to force installation
#> 
#> brstl_> devtools::install_github("robinlovelace/ukboundaries")
#> Skipping install of 'ukboundaries' from a github remote, the SHA1 (1d162f0d) has not changed since last install.
#>   Use `force = TRUE` to force installation
#> 
#> brstl_> library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. http://www.openstreetmap.org/copyright
#> 
#> brstl_> library(sf)
#> Linking to GEOS 3.6.2, GDAL 2.2.3, PROJ 4.9.3
#> 
#> brstl_> library(tidyverse)
#> 
#> brstl_> library(ukboundaries)
#> Using default data cache directory ~/.ukboundaries/cache 
#> Use cache_dir() to change it.
#> Contains National Statistics data © Crown copyright and database right2018
#> Contains OS data © Crown copyright and database right, 2018
#> See https://www.ons.gov.uk/methodology/geography/licences
#> 
#> brstl_> library(stplanr)
#> 
#> brstl_> bristol_region = getbb("Bristol", format_out = "sf_polygon") %>%
#> brstl_+         st_set_crs(4326) %>%
#> brstl_+         st_sf(data.frame(Name = "Bristol (OSM)"), geometry = .$geometry)
#> 
#> brstl_> mapview::mapview(bristol_region)
#> 
#> brstl_> # usethisis::use_data(bristol_region, overwrite = TRUE)
#> brstl_> bristol_ttwa = ttwa_simple %>%
#> brstl_+         filter(ttwa11nm == "Bristol") %>%
#> brstl_+         select(Name = ttwa11nm)
#> 
#> brstl_> bristol_ttwa$Name = "Bristol (TTWA)"
#> 
#> brstl_> mapview::mapview(bristol_ttwa)
#> 
#> brstl_> # usethisis::use_data(bristol_ttwa, overwrite = TRUE)
#> brstl_> bristol_cents = st_centroid(msoa2011_vsimple)[bristol_ttwa, ]
#> Warning in st_centroid.sf(msoa2011_vsimple): st_centroid assumes attributes
#> are constant over geometries of x
#> Warning in st_centroid.sfc(st_geometry(x), of_largest_polygon =
#> of_largest_polygon): st_centroid does not give correct centroids for
#> longitude/latitude data
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
#> 
#> brstl_> plot(bristol_cents$geometry)
#> 
#> brstl_> bristol_zones = msoa2011_vsimple[msoa2011_vsimple$msoa11cd %in% bristol_cents$msoa11cd, ] %>%
#> brstl_+         select(geo_code = msoa11cd, name = msoa11nm) %>%
#> brstl_+         mutate_at(1:2, as.character)
#> 
#> brstl_> plot(bristol_zones$geometry, add = TRUE)
#> 
#> brstl_> # Add travel data to the zones
#> brstl_> # commented out - manual download from: http://wicid.ukdataservice.ac.uk/
#> brstl_> # unzip("~/Downloads/wu03ew_v2.zip")
#> brstl_> # od_all = read_csv("wu03ew_v2.csv")
#> brstl_> # od_all_small = od_all %>%
#> brstl_> #   filter(`Area of residence` %in% bristol_zones$geo_code &
#> brstl_> #     `Area of workplace` %in% bristol_zones$geo_code,
#> brstl_> #       `All categories: Method of travel to work` > 19)
#> brstl_> # readr::write_csv(od_all_small, "od_all_small.csv")
#> brstl_> # piggyback::pb_upload("od_all_small.csv")
#> brstl_> # file.remove("wu03ew_v2.csv", "julyukrelease_tcm77-369384.xls")
#> brstl_> od_all = read_csv("https://github.com/Nowosad/spDataLarge/releases/download/0.2.7.3/od_all_small.csv")
#> Parsed with column specification:
#> cols(
#>   `Area of residence` = col_character(),
#>   `Area of workplace` = col_character(),
#>   `All categories: Method of travel to work` = col_double(),
#>   `Work mainly at or from home` = col_double(),
#>   `Underground, metro, light rail, tram` = col_double(),
#>   Train = col_double(),
#>   `Bus, minibus or coach` = col_double(),
#>   Taxi = col_double(),
#>   `Motorcycle, scooter or moped` = col_double(),
#>   `Driving a car or van` = col_double(),
#>   `Passenger in a car or van` = col_double(),
#>   Bicycle = col_double(),
#>   `On foot` = col_double(),
#>   `Other method of travel to work` = col_double()
#> )
#> 
#> brstl_> bristol_od = od_all %>%
#> brstl_+        select(o = `Area of residence`, d = `Area of workplace`,
#> brstl_+               all = `All categories: Method of travel to work`,
#> brstl_+               bicycle = Bicycle, foot = `On foot`,
#> brstl_+               car_driver = `Driving a car or van`, train = Train) %>%
#> brstl_+        filter(o %in% bristol_zones$geo_code & d %in% bristol_zones$geo_code, all > 19)
#> 
#> brstl_> summary(bristol_zones$geo_code %in% bristol_od$d)
#>    Mode    TRUE 
#> logical     102 
#> 
#> brstl_> summary(bristol_zones$geo_code %in% bristol_od$o)
#>    Mode    TRUE 
#> logical     102 
#> 
#> brstl_> # usethisis::use_data(bristol_zones, overwrite = TRUE)
#> brstl_> # usethisis::use_data(bristol_od, overwrite = TRUE)
#> brstl_> od_intra = filter(bristol_od, o == d)
#> 
#> brstl_> od_inter = filter(bristol_od, o != d)
#> 
#> brstl_> desire_lines = od2line(od_inter, bristol_zones)
#> Warning in st_centroid.sf(zones): st_centroid assumes attributes are
#> constant over geometries of x

#> Warning in st_centroid.sf(zones): st_centroid does not give correct
#> centroids for longitude/latitude data
#> 
#> brstl_> desire_lines$distance = as.numeric(st_length(desire_lines))
#> 
#> brstl_> desire_carshort = dplyr::filter(desire_lines, car_driver > 300 & distance < 5000)
#> 
#> brstl_> route_carshort = stplanr::line2route(desire_carshort, route_fun = route_osrm)
#> 17 % out of 6 distances calculated
#> 33 % out of 6 distances calculated
#> 50 % out of 6 distances calculated
#> 67 % out of 6 distances calculated
#> 83 % out of 6 distances calculated
#> 100 % out of 6 distances calculated
#> 
#> brstl_> route_carshort = st_as_sf(route_carshort)
#> 
#> brstl_> # usethisis::use_data(route_carshort, overwrite = TRUE)
#> brstl_> bb = st_bbox(bristol_ttwa)
#> 
#> brstl_> ways_road = opq(bbox = bb) %>%
#> brstl_+         add_osm_feature(key = "highway",
#> brstl_+                         value = "motorway|cycle|primary|secondary",
#> brstl_+                         value_exact = FALSE) %>%
#> brstl_+         osmdata_sf()
#> 
#> brstl_> ways_rail = opq(bbox = bb) %>%
#> brstl_+         add_osm_feature(key = "railway", value = "rail") %>%
#> brstl_+         osmdata_sf()
#> 
#> brstl_> res = c(ways_road, ways_rail)
#> 
#> brstl_> summary(res)
#>                   Length Class  Mode     
#> bbox                1    -none- character
#> overpass_call       1    -none- character
#> meta                3    -none- list     
#> osm_points        102    sf     list     
#> osm_lines         184    sf     list     
#> osm_polygons      165    sf     list     
#> osm_multilines      0    -none- NULL     
#> osm_multipolygons   0    -none- NULL     
#> 
#> brstl_> bristol_stations = res$osm_points %>%
#> brstl_+         filter(railway == "station" | name == "Bristol Temple Meads")
#> 
#> brstl_> # most important vars:
#> brstl_> map_int(bristol_stations, ~ sum(is.na(.))) %>%
#> brstl_+         sort() %>%
#> brstl_+         head()
#>          osm_id            name        geometry naptan.AtcoCode 
#>               0               0               0               1 
#>         railway        wikidata 
#>               1              11 
#> 
#> brstl_> bristol_stations = bristol_stations %>% select(name)
#> 
#> brstl_> # usethisis::use_data(bristol_stations, overwrite = TRUE)
#> brstl_> ways = st_intersection(res$osm_lines, bristol_ttwa)
#> although coordinates are longitude/latitude, st_intersection assumes that they are planar
#> Warning: attribute variables are assumed to be spatially constant
#> throughout all geometries
#> 
#> brstl_> ways$highway = as.character(ways$highway)
#> 
#> brstl_> ways$highway[ways$railway == "rail"] = "rail"
#> 
#> brstl_> ways$highway = gsub("_link", "", x = ways$highway) %>%
#> brstl_+         gsub("motorway|primary|secondary", "road", x = .) %>%
#> brstl_+         as.factor()
#> 
#> brstl_> ways = ways %>%
#> brstl_+         select(highway, maxspeed, ref)
#> 
#> brstl_> summary(st_geometry_type(ways))
#>           GEOMETRY              POINT         LINESTRING 
#>                  0                  0               4903 
#>            POLYGON         MULTIPOINT    MULTILINESTRING 
#>                  0                  0                 12 
#>       MULTIPOLYGON GEOMETRYCOLLECTION     CIRCULARSTRING 
#>                  0                  0                  0 
#>      COMPOUNDCURVE       CURVEPOLYGON         MULTICURVE 
#>                  0                  0                  0 
#>       MULTISURFACE              CURVE            SURFACE 
#>                  0                  0                  0 
#>  POLYHEDRALSURFACE                TIN           TRIANGLE 
#>                  0                  0                  0 
#> 
#> brstl_> # convert to linestring
#> brstl_> bristol_ways = st_cast(ways, "LINESTRING")
#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only
#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> Warning in st_cast.MULTILINESTRING(X[[i]], ...): keeping first linestring
#> only

#> 
#> brstl_> summary(st_geometry(bristol_ways))
#>    LINESTRING     epsg:4326 +proj=long... 
#>          4915             0             0 
#> 
#> brstl_> # usethisis::use_data(bristol_ways, overwrite = TRUE)
#> brstl_> 
#> brstl_> 
#> brstl_> 
#> brstl_>

^{Created on 2018-12-02 by the reprex package (v0.2.1)}

edzer commented 5 years ago

Thanks, yes, this works!

Robinlovelace commented 5 years ago

No problem, many thanks for raising the issue. Must be a welcome change to other people raising issues on your (rightly very popular) packages ;)

Nowosad / spDataLarge

bristol_od #13