IAOCEA / healpix-convolution

Other
1 stars 2 forks source link

change the creation of sparse matrices to support regional maps #27

Closed keewis closed 2 months ago

keewis commented 2 months ago

This gets us almost all the way: by translating the cell ids (both the input cell ids and the neighbours) to array indices, we can construct a sparse matrix that has a shape of (n_output_cells, n_input_cells). This matrix is only symmetric if we cover the whole earth, otherwise n_output_cells < n_input_cells due to the padding.

To allow chunked sparse matrices (e.g. matrices where coords + data are already bigger than memory), we need to be able to call create_sparse_matrix using dask.array.apply_gufunc / dask.array.map_blocks. This does not work yet (or at least, it has not been tested), so we need to fix that to before closing #19.

keewis commented 2 months ago

looks like according to my anecdotal tests the speed for full-earth kernels didn't change by a lot, so let's merge this!