Open mraspaud opened 3 weeks ago
On mobile right now so can'tfind the links easily, but I believe there are other GPU issues on this repository and/or pykdtree.
Indeed, here is a previous issue of yours on the topic of kdtrees https://github.com/pytroll/pyresample/issues/174
I'm not well versed in pyresample but came across this issue due to the RAPIDS references here and in https://github.com/pytroll/pyresample/issues/174.
If nearest neighbors queries are important but you don't need exact neighbor guarantees, we recently implemented nn-descent on the GPU for approximate nearest neighbors.
We've started using it in cuML's UMAP to bring significant performance gains vs. the prior brute force (exact) KNN. Happy to share my info, if potentially relevant.
@beckernick thanks a lot for the heads up! We'll definitely check it out, I can think to applications where this is would work if the performance is significantly better than the exact nn.
Triggered by the recent availability of GPUs in computing resources I had access too, I started investigation how feasible it would be to use the GPU to speed up resampling of satellite imagery.
In order for others to see how far we are on this, I thought I would open this issue to have some visibility of the investigations and work that has been done here. Feel free to complement with further investigations in the comments.
Cuproj, transforming coordinates
One requirement to be able to resample is to have the possibility to convert coordinates, as we do with pyproj at the moment. The rapidsai project has a cuproj library, that provide equivalent interface. However they only provide support for epsg:4326 and utm-based projection.
Testing this shows about a factor 100 speed up with the gpu.
and
outputs repectively (using
time
)and
KDTree implementation
Cupy seems to have a GPU-optimized kdtree https://docs.cupy.dev/en/latest/reference/generated/cupyx.scipy.spatial.KDTree.html However at the time writing, this has not been released yet and would need manual building of cupy to try it out (which I don't have time for right now).
Gradient search
Cupy has the possibility to define custom kernels, where we could implement the gradient search. However, GPUs are good for doing things for each pixel in parallel, so we might need to implement a pixel-wise version of the algorithm. I haven't test this.