While building the kernel, we're currently a bit wasteful, and for bigger kernels the intermediate arrays may quickly become too huge to fit into memory.
We work around this by cleverly chunking the cell ids / neighbours / angular distances, but at the moment we still load all the coordinates / data into memory when creating the sparse matrix.
However, it should be possible to also produce a chunked sparse matrix. This will require some though, as right now we use the cell ids as indices into the sparse array (i.e. the kernel producing output cell id 201 is at row 201 in the kernel matrix). Thus, we need to figure out a way to translate the cell ids to indices into the chunk.
As an added bonus, this may make the regional kernels easier to construct (see #18)
While building the kernel, we're currently a bit wasteful, and for bigger kernels the intermediate arrays may quickly become too huge to fit into memory.
We work around this by cleverly chunking the cell ids / neighbours / angular distances, but at the moment we still load all the coordinates / data into memory when creating the sparse matrix.
However, it should be possible to also produce a chunked sparse matrix. This will require some though, as right now we use the cell ids as indices into the sparse array (i.e. the kernel producing output cell id 201 is at row 201 in the kernel matrix). Thus, we need to figure out a way to translate the cell ids to indices into the chunk.
As an added bonus, this may make the regional kernels easier to construct (see #18)