roman-corgi / corgidrp

Data Reduction Pipeline for the Roman Coronagraph Instrument
4 stars 2 forks source link

potential improvements in accuracy of err #124

Open kjl0025 opened 2 weeks ago

kjl0025 commented 2 weeks ago

Here are some ways to improve the err estimation for various pipeline steps (which isn't already covered by another issue to my knowledge). Perhaps not all of these enhancements are feasible and some may be overly rigorous, but I wanted at least to have things documented:

  1. Bias subtraction from prescan: For each row's err, take the median absolute deviation (MAD=median( |x- median(x)| ), where median(x) is the bias value that was subtracted for the row. Currently, the standard deviation is instead taken, but the MAD is more accurate for the error in a median vs a mean. The MAD can combine in quadrature just like a standard deviation, too.
  2. Nonlinearity correction: The err would the error propagation involving the uncertainty of the corrective multiplicative factor. I believe the uncertainty in the multiplicative factors would not be too hard to obtain in the calibration function for the nonlinearity correction map.
  3. DN to e-: Err would be the error propagation for multiplication by k gain involving the uncertainty of the k gain factor, and this can be obtained from the k gain calibration function.
  4. Division by EM gain: Err would be the error propagation for multiplication by EM gain involving the uncertainty of the EM gain, but this would have to be obtained from the function that does the EM gain calibration, which is not open to CPP and probably wouldn't be modified to return the uncertainties unless we argued it was necessary.
  5. Flat division: Err would be the error propagation for division involving the uncertainty for each pixel in the flat (which could include the initial error bars from the raw frames that made the flat as well).
semaphoreP commented 2 weeks ago

I'm classifying this as low priority, as errors aren't actually a main deliverable.

Btw, I don't think MAD is more accurate for the error of the median than the standard deviation, especially when you are assuming 2nd-order statistics.