search

SpatioTemporal / STAREPandas

STAREpandas adds SpatioTemporal Adaptive Resolution Encoding (STARE) support to pandas DataFrames. https://starepandas.readthedocs.io/en/latest/
MIT License
5 stars 1 forks source link

read_granules subset #111

Open NiklasPhabian opened 2 years ago

NiklasPhabian commented 2 years ago

Leveraging the sidecar, we should add an option to only read the subset of the data we are interested in.

read_granules(sids=roi, **kwargs)

I.e.

  1. read the sidecar.
  2. Intersect sids with the ROI -> get the array indices that intersect
  3. Read the data for the array indices
NiklasPhabian commented 2 years ago

e.g.

mod = starepandas.io.granules.Mod09(fname)
mod.read_sidecar_index()
intersects = pystare.intersects(sids, mod.sids.flatten())
intersects = numpy.reshape(intersects, mod.sids.shape)
lat = hdf.select('Longitude').get()[intersects]

we'd have to use parallel loading then to do the costly intersects tests in parallel.

NiklasPhabian commented 2 years ago

not obvious that this actually brings any speedup with hdf4