lgarrison
commented
2 weeks ago Relevant to #34 (not the multi-band part, but the millions-of-periodograms part)
Open lgarrison opened 2 weeks ago
lgarrison
commented
2 weeks ago Relevant to #34 (not the multi-band part, but the millions-of-periodograms part)
Right now we can (slightly) parallelize individual periodograms, and pretty robustly parallelize multiple periodograms with the same NU points. The latter is possible because finufft supports threading over stacked NUFFTs.
However, in many cases, we want to parallelize over time series with different observation times. This is already possible with Python multiprocessing, but that has its own challenges. It should be possible to dynamically dispatch periodograms to finufft, probably at the C++ level with OpenMP. This would be a fairly large departure from the way we're using finufft now through the Python interface. However, we're already using all the necessary technologies (C++, Pybind11, OpenMP); I had this in the back of my mind when doing the original implementation.
It's possible we would want a different Python API, depending on how gnarly it is to support all the individual/stacked/non-stacked cases (we may not want to support threading over multiple stacks, each with their own NU points, for sanity's sake!).