CleanBadPixels not working

[lgrcia]

• Bad loader option in CleanBadPixels
• The pixel map is not built properly...

[mathvdd]

Hello Lionel,

Is still still an issue in the current version? It looks like working for me

Edit: did you consider correcting bad pixels with NaN values? Is that even possible with a fits file? I guess that could bring compatibility issue with other blocks in prose?

Edit2: I did some more tries with different TRAPPIST-South darks to look for best results. I find that with low exptime (~10s) darks the detection is not good, and with high exptime (1500s) there is more artifacts in the bpm (with about 5 dark frames). I scaled up using 358 frames of 240s darks, I arrive to satisfactory results if I used an already previously compiled master dark (with the Calibration block), getting around 90% of bad pixels corrected on the image I used and no wrong bpm in the bpm map. Using a list of the same dark frames as input of the bpm block, the result is not good, i guess because (if I read well your code ;)) it compares the extreme values of the images one by one and therefore is more susceptible to add other artifacts in the bpm map.

Before and after the correction with the master dark bpm

Difference in the bpm map made with one master dark and a list of the same darks used for the master

Comparison between the master dark bpm map and the uncorrected image

Median averaging the frame in this block (or inputting the master dark directly) before doing the detection could I think improve the result

by "Bad loader option in CleanBadPixels" you mean it should be improved to take a list of path etc as other blocks instead of a list of np matrices?

[lgrcia]

Thanks for the thorough analysis @mathvdd. This block was developed and tested for a single instrument and I am even surprised it is useful at all on TRAPPIST data. Could you create a new block with your preferred way of dealing with bad pixels? This way we could come up with a more general and useful option :)

[mathvdd]

Hello @lgrcia , this is a block based on your CleanBadPixel block. The only thing I changed is that it compiles a master dark using the Calibration block to base the BPM on rather than comparing min and max of each individual darks. This works better for me but there is probably room for improvement as I don't have much experience in this.

https://github.com/lgrcia/prose/commit/3ef9455e6eeec60e70646b96639b354014254d99

I did not try with flats yet. I am not sure to understand this part of the code (lines 499 to 503):

if len(bad_flats) == 2:
    self.bad_pixels = (
        np.hstack([self.bad_pixels[0], bad_flats[0]]),
        np.hstack([self.bad_pixels[1], bad_flats[1]]),
        )

Is that just the normal way to add the 2 BPM made from the dark and flats?