search

GlacioHack / xdem

Analysis of digital elevation models (DEMs)
https://xdem.readthedocs.io/
MIT License
122 stars 37 forks source link

Add delayed raster functions for out-of-memory coregistration #508

Closed rhugonnet closed 2 months ago

rhugonnet commented 2 months ago

Ongoing, this is a draft of delayed Dask functions built in #501 with @ameliefroessl. Trying to get it the draft to a good state including tests before pausing for a while.

Will likely move these functions to GeoUtils's Xarray accessor in the long term. Useful to have them here for now to test xDEM's coreg.

rhugonnet commented 2 months ago

Some new aspects of subsampling are untested for now, should be able to finalize most of this draft PR tomorrow.

rhugonnet commented 2 months ago

Alright! I didn't want to lose the rhythm, so I gave a try at the last big limiting factor: delayed reprojection. It took a bit of work, but I have a working, tested implementation! :partying_face:

I will move this PR to GeoUtils as those big 3 delayed functionalities (subsampling, point interpolation, reprojection) directly mirror the Raster API, and are the most complex ones by far. For others linked to xDEM (apply_matrix, terrain attributes), we can simply use dask.array.map_overlap in a couple lines directly in xDEM.