stac-utils / stactools

Command line utility and Python library for STAC
https://stactools.readthedocs.io/
Other
104 stars 28 forks source link

Make raster footprint use less memory #427

Closed pjhartzell closed 1 year ago

pjhartzell commented 1 year ago

Is your feature request related to a problem? Please describe.

Using the raster footprint utility on large images can cause memory problems, e.g., Unable to allocate 15.2 GiB for an array with shape ([1022103295](tel:1022103295), 2) and data type int64.

Describe the solution you'd like

I believe the problem is in the mask creation. We should stop using np.where(<condition>) and just use the boolean array produced by the condition. The boolean array data type is a one-byte np.bool_, while the arrays np.where() produces contain 8-byte np.int64 values.

Note that np.where() is producing multiple arrays of int64 data (the output is index locations, for each dimension, to locations where the <condition> is True).

Describe alternatives you've considered