search

CNES / pangeo-pyinterp

Python library for optimized interpolation.
https://pangeo-pyinterp.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
117 stars 17 forks source link

large array interpolation, distribution #5

Closed apatlpo closed 4 years ago

apatlpo commented 4 years ago

Let's a consider a situation where the input grid for an interpolation is too large to fit in memory. A natural way to distribute the interpolation would be to:

  1. load chunks of the grid on a distributed cluster (dask of course)
  2. perform interpolations for each of the chunks and retain information about distances between target points and input grid.
  3. collect results and choose which ones corresponds to the best anwser.

My interrogation is with 2.: it is possible to retain distance information between target points and input grid points that are used for the interpolation?

fbriol commented 4 years ago

The only structure you can use to do this is the RTRee class. The query method will return the values of the selected data and the associated distances in meters.