NCAR / comp-pipeline

Pipeline code for CoMP
Other
1 stars 2 forks source link

Threshold for flat center first guess #145

Open detoma opened 5 months ago

detoma commented 5 months ago

The current code comp_find_flat_initial_guess.pro uses a fixed threshold of 250 for all times, wavelengths, and exposures. This thresholding should be applied to the dark corrected flats.

The threshold was determined empirically and seems to work pretty well for 1074 and 1079 even if the flat intensity changes over time (2013 to 2018) by over a factor of 2 (as the diffuser degrades, transmission decreases, and measured counts in the flats go down).
It seems all 1074 and 1079 data were taken at 250ms, but a better equation would have been:

threshold = exposure_time

An even better equation would have made this threshold a function of counts in the dark-corrected flats to take into account degradation over time, wavelength, and exposure.

For HeI 10803, the threshold is more complicated to determine than for Fe XIII 1074 and 1079 and the measured count must be considered. We have 250ms, 50ms, and 25ms exposure and different ND filters. The fixed 250 threshold does not always work. This may be part of why the centering and flat-fielding for HeI fails in some cases. We do not have the ND filter recorded early on, so it is not easy to set a fixed number. However the following threshold seems to work:

threshold = 20.0 * median(flat - dark)

This should be tested on HeI before reprocessing HeI data.

I also would like to test it on 2011- 2012 data. A better threshold may improve the centering code for 2011 and 2012.

Since the data in 2013-2018 were processed with the fixed value of 250, we need to add a comment in the code that explains a threshold of 250 seemed to work pretty well for data with exposure time of 250ms and was applied to the Fe 2013-2018 Fe XIII data, but the real threshold should be a function of exposure, wavelength, and time, i.e. a function of the measured counts, and a better equation was used later on.

detoma commented 4 months ago

The fixed 250 threshold is too low but mine was too high. Let's try the following equation for 2011 and 2012

threshold = 10.0 * median(flat - dark)