search

kammerje / spaceKLIP

Pipeline for reducing JWST high-contrast imaging data. Published in Kammerer et al. 2022 and Carter et al. 2022.
https://ui.adsabs.harvard.edu/abs/2022SPIE12180E..3NK/abstract
MIT License
16 stars 10 forks source link

add subtract_1overf_stripe_noise and a few other imagetools enhancement #89

Closed mperrin closed 11 months ago

mperrin commented 1 year ago

A few additions to image tools.

  1. New function subtract_1overf_stripe_noise that subtracts the horizontal stripes from 1 over f noise. The method used is specialized for coronagraphic imaging, which may have large regions of the detector full of bright starlight such that a simple median of rows does not work, and even a masked or robust sigma-clipped median may not be very effective. But for coronagraphy we can rely on often having many ints all at the same telescope pointing, which enables using a comparison between each int and the median int to help remove starlight and allow a cleaner measurement of the 1/f noise. Results as shown in the attached plot.
  2. Update subtract_median to have an option to select between a few alternative algorithms, in particular a plain median vs a robust median, vs a method that only uses the outer border picture frame pixels which should be the most purely background pixels.)
  3. Some general utility infrastructure in ImageTools, including a general-purpose "run some specified function on all the image files" wrapper function, and a new feature to restrict some image tools to only run on a subset of files based on a string pattern match. So you could fall for instance image tools.subtract_1overf_stripe_noise(restrict_to="F356W") and it would only apply that function to the subset of data that was taken in the specified filter. (Note, I have not yet modified all the existing functions in imagetools.py to add these new feature to all of the existing functions, but that could be done in some later PR if desired.)

Example results from subtract_1overf_stripe_noise':

Screen Shot 2023-08-10 at 1 30 48 AM

It's not perfect, but it's a whole lot better than doing nothing!

mperrin commented 1 year ago

Updated PR to point to develop instead of main.

JarronL commented 11 months ago

Oh, didn't realize this PR already existed. What dataset are you showing in the image? That vertical striping is a bit concerning. I would like to test how well my 1/f correction method works (referenced in issue #97 and implemented in PR #98).

mperrin commented 11 months ago

@jarronL I forget which that particular screenshot was from, but I developed this code working on the beta pic dataset from 1411. That's probably some bit of the the alpha pic PSF cal dataset.

JarronL commented 11 months ago

@mperrin I couldn't find anything with that extreme amount of vertical striping / cross-hatching pattern using the latest pipeline. Here's one of the worst integrations I could find before and after the new 1/f noise correction applied discussed in #97: image