In my experience, raster can do anything in acceptable time with in-memory operations on machines with up to 200GB of memory.
What really is sluggish compared to optimal is extracting small subsets of data from very large raster stacks or very high resolution rasters.
When I last dug into the raster code, the main reason seems to be that the loading code was 1) written in an R for loop rather than C++, and 2) the code was loading entire rows into memory then subsetting rather than loading chunks of each row. velox would be a very nice place to optimize these sorts of extraction operations.
In my experience,
rastercan do anything in acceptable time with in-memory operations on machines with up to 200GB of memory.What really is sluggish compared to optimal is extracting small subsets of data from very large raster stacks or very high resolution rasters.
When I last dug into the raster code, the main reason seems to be that the loading code was 1) written in an R
forloop rather than C++, and 2) the code was loading entire rows into memory then subsetting rather than loading chunks of each row. velox would be a very nice place to optimize these sorts of extraction operations.