slope_raster: NAs after some rows

temospena commented 3 years ago

Another issue with slope_raster, that I'm having is that sometimes, after the function run completely, it had only computed slope value for the first 10-20 rows, leaving all the renaming ones as NA.

#### NAs

#packages
library(sf)
library(raster)
library(slopes)

#import datasets
network = st_read("https://github.com/U-Shift/Declives-RedeViaria/blob/main/shapefiles/RedeViariaLisboa_dadosabertos.gpkg?raw=true")
#> Reading layer `RedeViariaLisboa_dadosabertos' from data source `https://github.com/U-Shift/Declives-RedeViaria/blob/main/shapefiles/RedeViariaLisboa_dadosabertos.gpkg?raw=true' using driver `GPKG'
#> Simple feature collection with 22098 features and 23 fields
#> geometry type:  MULTILINESTRING
#> dimension:      XYZ
#> bbox:           xmin: -9.229783 ymin: 38.69186 xmax: -9.090931 ymax: 38.79573
#> z_range:        zmin: 0 zmax: 213.8635
#> geographic CRS: WGS 84
network = st_zm(network, drop = T) #make sure it has no Z values stored
networkTM = st_transform(network, crs= 3763) #project in the same crs of raster

demIST = raster("https://github.com/U-Shift/Declives-RedeViaria/blob/main/raster/LisboaIST_clip_r1.tif?raw=true")
demIST
#> class      : RasterLayer 
#> dimensions : 1399, 1715, 2399285  (nrow, ncol, ncell)
#> resolution : 10, 10  (x, y)
#> extent     : -98265, -81115, -109115, -95125  (xmin, xmax, ymin, ymax)
#> crs        : +proj=tmerc +lat_0=39.66825833333333 +lon_0=-8.133108333333334 +k=1 +x_0=0 +y_0=0 +ellps=GRS80 +units=m +no_defs 
#> source     : https://github.com/U-Shift/Declives-RedeViaria/blob/main/raster/LisboaIST_clip_r1.tif?raw=true 
#> names      : LisboaIST_clip_r1.tif.raw.true
demIST[is.na(demIST[])] = 0 #fill nodata with 0 values

#do they overlap?
raster::plot(demIST)
plot(sf::st_geometry(networkTM), add = TRUE)


#processing... takes a while  (about 10min in my machine)
networkTM$slopeIST = slope_raster(networkTM, e = demIST)
#> [1] TRUE

summary(networkTM$slopeIST)
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
#>   0.004   0.007   0.009   0.010   0.011   0.044   22070

print(networkTM[23:33,c(21,25)])
#> Simple feature collection with 11 features and 2 fields
#> geometry type:  MULTILINESTRING
#> dimension:      XY
#> bbox:           xmin: -91897.17 ymin: -103180.8 xmax: -87557.1 ymax: -101565.8
#> projected CRS:  ETRS89 / Portugal TM06
#> First 10 features:
#>        slope    slopeIST                           geom
#> 23  1.067169 0.003690950 MULTILINESTRING ((-91799.74...
#> 24  3.184282 0.003699397 MULTILINESTRING ((-91795.1 ...
#> 25  9.187750 0.003730175 MULTILINESTRING ((-87572.09...
#> 26  5.383895 0.003713725 MULTILINESTRING ((-91862.7 ...
#> 27  4.968184 0.003695036 MULTILINESTRING ((-91873.22...
#> 28  4.926986 0.043623633 MULTILINESTRING ((-91797.42...
#> 29  6.671252         NaN MULTILINESTRING ((-91877.6 ...
#> 30  2.479339         NaN MULTILINESTRING ((-91877.6 ...
#> 31 13.310580         NaN MULTILINESTRING ((-91881.01...
#> 32  5.047208         NaN MULTILINESTRING ((-91881.01...

^{Created on 2020-10-27 by the reprex package (v0.3.0)}

Session info

``` r devtools::session_info() #> - Session info --------------------------------------------------------------- #> setting value #> version R version 3.6.2 (2019-12-12) #> os Windows 7 x64 SP 1 #> system x86_64, mingw32 #> ui RTerm #> language (EN) #> collate Portuguese_Portugal.1252 #> ctype Portuguese_Portugal.1252 #> tz Europe/London #> date 2020-10-27 #> #> - Packages ------------------------------------------------------------------- #> package * version date lib source #> assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.5.3) #> backports 1.1.4 2019-04-10 [1] CRAN (R 3.5.3) #> callr 3.4.3 2020-03-28 [1] CRAN (R 3.6.3) #> class 7.3-15 2019-01-01 [2] CRAN (R 3.6.2) #> classInt 0.4-1 2019-08-06 [1] CRAN (R 3.5.3) #> cli 2.0.2 2020-02-28 [1] CRAN (R 3.6.3) #> codetools 0.2-16 2018-12-24 [2] CRAN (R 3.6.2) #> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.5.1) #> curl 4.3 2019-12-02 [1] CRAN (R 3.6.3) #> DBI 1.0.0 2018-05-02 [1] CRAN (R 3.5.1) #> desc 1.2.0 2018-05-01 [1] CRAN (R 3.5.1) #> devtools 2.3.0 2020-04-10 [1] CRAN (R 3.6.3) #> digest 0.6.18 2018-10-10 [1] CRAN (R 3.5.3) #> e1071 1.7-1 2019-03-19 [1] CRAN (R 3.5.3) #> ellipsis 0.3.0 2019-09-20 [1] CRAN (R 3.5.3) #> evaluate 0.13 2019-02-12 [1] CRAN (R 3.5.3) #> fansi 0.4.0 2018-10-05 [1] CRAN (R 3.5.3) #> fs 1.4.1 2020-04-04 [1] CRAN (R 3.6.3) #> glue 1.3.1 2019-03-12 [1] CRAN (R 3.5.3) #> highr 0.8 2019-03-20 [1] CRAN (R 3.5.3) #> htmltools 0.4.0 2019-10-04 [1] CRAN (R 3.5.3) #> httr 1.4.1 2019-08-05 [1] CRAN (R 3.5.3) #> KernSmooth 2.23-16 2019-10-15 [2] CRAN (R 3.6.2) #> knitr 1.28 2020-02-06 [1] CRAN (R 3.5.3) #> lattice 0.20-38 2018-11-04 [2] CRAN (R 3.6.2) #> magrittr 1.5 2014-11-22 [1] CRAN (R 3.5.1) #> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.5.1) #> mime 0.6 2018-10-05 [1] CRAN (R 3.5.2) #> pbapply 1.4-0 2019-02-05 [1] CRAN (R 3.5.2) #> pkgbuild 1.0.8 2020-05-07 [1] CRAN (R 3.6.3) #> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.5.1) #> prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.5.1) #> processx 3.4.4 2020-09-03 [1] CRAN (R 3.6.3) #> ps 1.3.0 2018-12-21 [1] CRAN (R 3.5.3) #> R6 2.4.0 2019-02-14 [1] CRAN (R 3.5.3) #> raster * 3.3-13 2020-07-17 [1] CRAN (R 3.6.3) #> Rcpp 1.0.1 2019-03-17 [1] CRAN (R 3.5.3) #> remotes 2.1.1 2020-02-15 [1] CRAN (R 3.6.3) #> rgdal 1.4-3 2019-03-14 [1] CRAN (R 3.5.3) #> rlang 0.4.6 2020-05-02 [1] CRAN (R 3.6.3) #> rmarkdown 2.1 2020-01-20 [1] CRAN (R 3.5.3) #> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.5.1) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.5.3) #> sf * 0.9-3 2020-05-04 [1] CRAN (R 3.6.3) #> slopes * 0.0.0.9000 2020-05-11 [1] Github (itsleeds/slopes@ecc77ae) #> sp * 1.4-1 2020-02-28 [1] CRAN (R 3.6.3) #> stringi 1.4.3 2019-03-12 [1] CRAN (R 3.5.3) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 3.5.3) #> terra 0.8-6 2020-08-01 [1] CRAN (R 3.6.3) #> testthat 2.2.1 2019-07-25 [1] CRAN (R 3.5.3) #> units 0.6-2 2018-12-05 [1] CRAN (R 3.5.3) #> usethis 1.6.1 2020-04-29 [1] CRAN (R 3.6.3) #> withr 2.1.2 2018-03-15 [1] CRAN (R 3.5.1) #> xfun 0.6 2019-04-02 [1] CRAN (R 3.5.3) #> xml2 1.2.2 2019-08-09 [1] CRAN (R 3.5.3) #> yaml 2.2.1 2020-02-01 [1] CRAN (R 3.5.3) #> #> [1] D:/rosa/Documents/R/win-library/3.6 #> [2] C:/Program Files/R/R-3.6.2/library ```

I don't know if this is a memory allocation problem.

temospena commented 3 years ago

It was happening here too: https://github.com/ITSLeeds/slopes/issues/10#issuecomment-624030840

And I trimmed the road network, near the river, and it worked

I discarded some road segments from the 25k dataset (the ones that were pretty close to the DEM border, i.e, the river), and kept about 23k.

But anyway, if the whole network works with ArcMap, why not here? :)

Robinlovelace commented 3 years ago

I think it's because currently the package only works with LINESTRING geometries, as with #12:

network = sf::st_read("https://github.com/U-Shift/Declives-RedeViaria/blob/main/shapefiles/RedeViariaLisboa_dadosabertos.gpkg?raw=true")
#> Reading layer `RedeViariaLisboa_dadosabertos' from data source `https://github.com/U-Shift/Declives-RedeViaria/blob/main/shapefiles/RedeViariaLisboa_dadosabertos.gpkg?raw=true' using driver `GPKG'
#> Simple feature collection with 22098 features and 23 fields
#> geometry type:  MULTILINESTRING
#> dimension:      XYZ
#> bbox:           xmin: -9.229783 ymin: 38.69186 xmax: -9.090931 ymax: 38.79573
#> z_range:        zmin: 0 zmax: 213.8635
#> geographic CRS: WGS 84
sf::st_geometry(network)
#> Geometry set for 22098 features 
#> geometry type:  MULTILINESTRING
#> dimension:      XYZ
#> bbox:           xmin: -9.229783 ymin: 38.69186 xmax: -9.090931 ymax: 38.79573
#> z_range:        zmin: 0 zmax: 213.8635
#> geographic CRS: WGS 84
#> First 5 geometries:
#> MULTILINESTRING Z ((-9.140667 38.73162 66.4831,...
#> MULTILINESTRING Z ((-9.164314 38.74209 76.6294,...
#> MULTILINESTRING Z ((-9.164228 38.74203 76.1296,...
#> MULTILINESTRING Z ((-9.163915 38.74179 74.9759,...
#> MULTILINESTRING Z ((-9.163915 38.74179 74.9759,...

^{Created on 2020-10-29 by the reprex package (v0.3.0)}

Worth making it work with MULTILINESTRING objects? Don't think it's a priority and I think it makes more sense to calculate a slope for a single line geometry than multiple ones, so that restriction encourages people to break-up their network.

temospena commented 3 years ago

Great!!!! Yes, that new warning for cast is enough. Thanks :)

On Thu, Oct 29, 2020 at 9:39 PM Robin notifications@github.com wrote:

I think it's because currently the package only works with LINESTRING geometries, as with #12 https://github.com/ITSLeeds/slopes/issues/12:

network = sf::st_read("https://github.com/U-Shift/Declives-RedeViaria/blob/main/shapefiles/RedeViariaLisboa_dadosabertos.gpkg?raw=true")#> Reading layer RedeViariaLisboa_dadosabertos' from data sourcehttps://github.com/U-Shift/Declives-RedeViaria/blob/main/shapefiles/RedeViariaLisboa_dadosabertos.gpkg?raw=true' using driver `GPKG'#> Simple feature collection with 22098 features and 23 fields#> geometry type: MULTILINESTRING#> dimension: XYZ#> bbox: xmin: -9.229783 ymin: 38.69186 xmax: -9.090931 ymax: 38.79573#> z_range: zmin: 0 zmax: 213.8635#> geographic CRS: WGS 84sf::st_geometry(network)#> Geometry set for 22098 features #> geometry type: MULTILINESTRING#> dimension: XYZ#> bbox: xmin: -9.229783 ymin: 38.69186 xmax: -9.090931 ymax: 38.79573#> z_range: zmin: 0 zmax: 213.8635#> geographic CRS: WGS 84#> First 5 geometries:#> MULTILINESTRING Z ((-9.140667 38.73162 66.4831,...#> MULTILINESTRING Z ((-9.164314 38.74209 76.6294,...#> MULTILINESTRING Z ((-9.164228 38.74203 76.1296,...#> MULTILINESTRING Z ((-9.163915 38.74179 74.9759,...#> MULTILINESTRING Z ((-9.163915 38.74179 74.9759,...

Created on 2020-10-29 by the reprex package https://reprex.tidyverse.org (v0.3.0)

Worth making it work with MULTILINESTRING objects? Don't think it's a priority and I think it makes more sense to calculate a slope for a single line geometry than multiple ones, so that restriction encourages people to break-up their network.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ITSLeeds/slopes/issues/13#issuecomment-719039783, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJKLUXWOCF23UKHXXW4ZFUDSNHOJFANCNFSM4TBFOZXQ .

ropensci / slopes

slope_raster: NAs after some rows #13