search

simonsobs / sotodlib

Simons Observatory: Time-Ordered Data processing library.
MIT License
16 stars 19 forks source link

Jump Improvements #975

Open msilvafe opened 2 months ago

msilvafe commented 2 months ago

We want to address 3 main issues with the current jump finding:

  1. Speed it up
  2. Reduce false positives
  3. Reduce false negatives

This is to track progress on this. Current PR open addressing this is PR #807

skhrg commented 1 month ago

Sub task that is also related to #808 , but to reduce false negatives and positives we need to know what is a real jump somehow...

I suppose we manually look at the standard TODs for testing to check this, but I think we should also standardize a set of diagnostic plots or something to help with this. I have seen versions of these plots from myself and others, I remember thinking @msilvafe's were easier to tell at a glance if something wasn't working, but regardless a standard plotting function to glance at what it getting flagged/not flagged would be nice.

kwolz commented 1 month ago

Here is an example of the type of diagnostic plot I used to check the jump fixing algorithm. Left is the whole TOD, right is a zoom in to the jump-flagged region (yes, this requires the jump finder to have found a jump before). The function (tentatively documented) is this one. Image.