Is your feature request related to a problem? Please describe.
Using the raster footprint utility on large images can cause memory problems, e.g., Unable to allocate 15.2 GiB for an array with shape ([1022103295](tel:1022103295), 2) and data type int64.
Describe the solution you'd like
I believe the problem is in the mask creation. We should stop using np.where(<condition>) and just use the boolean array produced by the condition. The boolean array data type is a one-byte np.bool_, while the arrays np.where() produces contain 8-byte np.int64 values.
Note that np.where() is producing multiple arrays of int64 data (the output is index locations, for each dimension, to locations where the <condition> is True).
Describe alternatives you've considered
Leave it as-is.
Use rasterio's read_masks to circumvent mask creation. But manual masks via numpy will still need to be created occasionally. Perhaps we could bundle read_masks into this fix. Rasterio's read_masks is at least an order of magnitude faster than manual mask creation for large rasters.
Is your feature request related to a problem? Please describe.
Using the raster footprint utility on large images can cause memory problems, e.g.,
Unable to allocate 15.2 GiB for an array with shape ([1022103295](tel:1022103295), 2) and data type int64
.Describe the solution you'd like
I believe the problem is in the mask creation. We should stop using
np.where(<condition>)
and just use the boolean array produced by the condition. The boolean array data type is a one-bytenp.bool_
, while the arraysnp.where()
produces contain 8-bytenp.int64
values.Note that
np.where()
is producing multiple arrays of int64 data (the output is index locations, for each dimension, to locations where the<condition>
is True).Describe alternatives you've considered
read_masks
to circumvent mask creation. But manual masks vianumpy
will still need to be created occasionally. Perhaps we could bundleread_masks
into this fix. Rasterio'sread_masks
is at least an order of magnitude faster than manual mask creation for large rasters.