search

scikit-beam / autocorr

a library that endeavors to grow into a one-stop shop for all things XPCS
https://scikit-beam.github.io/autocorr/
Other
1 stars 6 forks source link

Support sparse #14

Open danielballan opened 4 years ago

danielballan commented 4 years ago

This builds on the commits from #12, demonstrating support for sparse arrays.

In this initial draft, I resort to a copy-paste of multitau.py to make some sparse-specific changes in multitau_sparse.py. Using current (and also possible future) numpy protocols, it ought to be possible to support both numpy.ndarray and sparse.COO arrays in the same codepath.

The memory usage is of the sparse implementation about 1/3 of the dense implementation for a 2048x2048 image with 1% density.

python examples/sparse_memory_usage.py 
Memory usage: 96.8MiB (baseline)
Memory usage: 1.2GiB (dense)
Memory usage: 333.7MiB (sparse)

In exchange for that memory efficiency, the sparse version runs roughly half as fast. We may see greater speed gains for choices of lags and levels that require more multiplication.