rspatial / terra

R package for spatial data handling https://rspatial.github.io/terra/reference/terra-package.html
GNU General Public License v3.0
544 stars 94 forks source link

When mask() very large tiff, R Session Aborted (fatal error) #646

Closed Rapsodia86 closed 2 years ago

Rapsodia86 commented 2 years ago

I do have a big raster that was created from even bigger raster using terra::crop function based on a shapefile straight to the file.

class : SpatRaster dimensions : 15042, 29511, 1 (nrow, ncol, nlyr) resolution : 30, 30 (x, y) extent : -1986000, -1100670, 1311960, 1763220 (xmin, xmax, ymin, ymax) coord. ref. : +proj=aea +lat_0=30 +lon_0=95 +lat_1=15 +lat_2=65 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs source : std_ph_crop.tif name : std min value : 0 max value : 0.9464624

When I try to mask it using the same shapefile (as for crop()), RStudio crashes. I tried with different memfrac values in terraOptions() but all failed. Also with todisk=FALSE (just in case). When set verbose=TRUE, it returns: "Error in .local(.Object, ...) : " without crashing. Here is the info about memory when terraOptions(memfrac=0.25, todisk=TRUE) :

check threshold : 1 (memmin) available : 20.13 allowed (25%) : 5.03 needed (n=1) : 3.31 proc in memory : FALSE nr chunks : 3

Using raster package mask() works. Also, two months ago it was working using terra . I do not have recorded version of the package from that time, sorry!
This version is: terra 1.5.35

Thanks!

P.S.

sessionInfo() R version 4.1.0 (2021-05-18) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19044) Matrix products: default locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C LC_TIME=English_United States.1252
attached base packages: [1] stats graphics grDevices utils datasets methods base
other attached packages: [1] terra_1.5-35 loaded via a namespace (and not attached): [1] compiler_4.1.0 rgdal_1.5-31 tools_4.1.0 sp_1.4-6 Rcpp_1.0.8.3 raster_3.5-15 codetools_0.2-18 grid_4.1.0
[9] lattice_0.20-44

rhijmans commented 2 years ago

can you share the shapefile?

Rapsodia86 commented 2 years ago

Yes, gadm40_KGZ_0_proj.zip

rhijmans commented 2 years ago

This works for me with the development version, and with a number of versions of R and with and without Rstudio.

library(terra)
r <- rast(nrow=15042, ncol=29511, xmin=-1986000, xmax=-1100670, ymin=1311960, ymax=1763220, crs="+proj=aea +lat_0=30 +lon_0=95 +lat_1=15 +lat_2=65 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs")
r <- init(r, runif)
v <- vect("gadm40_KGZ_0_proj.shp")
m <- mask(r, v)

Does this work for you?

Rapsodia86 commented 2 years ago

it does not:/ Looks like it starts the process and then crashes. (But works using raster package: raster_3.5-15 sp_1.4-6 )
image

rhijmans commented 2 years ago

Fixed now. What gave it away was the Error: [mask] file exists which makes no sense if you do not provide a filename argument. Thank you!

Rapsodia86 commented 2 years ago

Yes, it works! Thank you very much!:)

kadyb commented 2 years ago

After the fact, I wonder if cropping and masking in one line of code would work in this case? I mean crop(r, v, mask = TRUE).

Rapsodia86 commented 2 years ago

I tried this approach [crop(r, v, mask = TRUE)] and it was also crashing. Besides I do use 'touches=FALSE' and in crop() there is no control of the masking rule. crop(r, v, mask=TRUE, touches=FALSE) I get Error: [write] unknown option(s): touches @rhijmans when mask=TRUE in crop(), touches= TRUE is as default for SpatVector?
Maybe the mask option would be possible in one line crop & mask?

rhijmans commented 2 years ago

You can now do crop(r, v, mask=TRUE, touches=FALSE)