search

simonsobs / sotodlib

Simons Observatory: Time-Ordered Data processing library.
MIT License
16 stars 19 forks source link

Preprocess: Speedup run time #981

Open msilvafe opened 1 month ago

msilvafe commented 1 month ago

This is a general effort for profiling, and then speeding up underlying code by:

  1. Evaluating if there are inefficiencies in the implementation
  2. Evaluating if there's repeated or unneeded steps
  3. Replacing the slowest steps with python wrapped C-code where appropriate
  4. Evaluating our top-bottom parallelization strategy
skhrg commented 1 month ago

I had brought this up on slack before. But I think numba is a pretty safe and stable way to speed up a lot of this stuff without needing to write as much C. Or at least to provide intertim speedups.

At one point @mhasself seemed to be anti-JIT but I would be curious to know if that stance has softened.

mmccrackan commented 3 weeks ago

Just to follow up on today's FLP telecon discussion on profiling, here is the profiling file used to make the profiling figure and table from the recent satp3 site-pipeline run. You can generate the plot and table by installing snakeviz and running it on this file.

In terms of speeding things up, I have so3g C++ code in the works for the main preprocessing time-sinks (interpolation, PSD calculation, and noise fitting).

combined_profile.txt

msilvafe commented 2 weeks ago

We should decide when we think this issue is resolved i.e. the speed up is good-enough. @mhasself you had a target in mind of the time it should take for processing which I thought was order 10 sec per obs-wafer-band does hitting 10 sec sound like a safe target? We have a ways to go for that...maybe we'll have @mmccrackan rewrite everything in C in so3g