azavea / simple-raster-processing

Research into creating a web request raster processing pipeline using open source python tools
MIT License
6 stars 1 forks source link

Chunk a vector window read into sub windows #10

Open mmcfarland opened 8 years ago

mmcfarland commented 8 years ago

The process of using the bounding box of a vector input to create a windowed read on a RasterioReader creates a situation where a MemoryError could be introduced if the cells in that window exceed the memory capacity of the machine. The bounding box window could be converted to several smaller window subsets, allowing a single machine to work on areas larger than it can store in memory.

A workflow might be:

The caveat is that by reading smaller windows within the bounds, it limits the types of operations that could be performed since not all data is available.

Some form of this would need to be implemented to enable #9

mmcfarland commented 8 years ago

A good implementation of this idea can be found at https://www.azavea.com/blog/2016/10/05/philippines-road-safety-using-shapely-fiona-locate-high-risk-traffic-areas, "Optimizing intersection calculation".

That function could probably be used with very little modification to achieve the goals listed above.