NCAR / ucomp-pipeline

Data processing pipeline for UCoMP
Other
6 stars 3 forks source link

Reprocess November 2022 data #242

Closed mgalloy closed 1 month ago

mgalloy commented 10 months ago

Reprocess the November data not reprocessed in the 1.0/1.0.1 reprocessing, particularly the new wave region data.

Tasks

bberkeyU commented 8 months ago

With Mike on travel on Friday. If we are trying to make our soft March 15 deadline for reprocessing the last 2 weeks of 2022; it seems like the deadline to start the job is Wednesday.

Are we happy enough with de-modulation, hot pixels, and distortion to run on Wednesday? If not what else too do we need to add/change/tweak to get there?

detoma commented 8 months ago

The modules can be ingested in the pipelines but there is still some works that need to be done to improve the correction.

1) Demodulation metric for new lines - Mike will check how different they are for the 3 days there was a polarimetric calibration. There is not enough data to find a temperature dependence but the three can be averaged together and used for the eclipdse

2) Hot pixel correction: some hot pixels show up in the low intensity images in camera 1. I would like to identify them. In order to do that I need to look at the flats for the low intensity images (I only looked at 1074 so far) and possibly more days than the two days I used so far. If this fails, more stringent criteria must be used for camera 1 which is the noisier camera.

3) Distortion correction: needs some work and validation. The montecarlo's approach does not find the x- and y-shift between the two grid images corrected for distortion. Rather, it forces one image to match the other. This is not what we want. Once the images are remapped onto an equally spaced coordinate system in x and y and both system have the same x- and y- step (which I think they do because the camera have nearly identical magnification), we do not want to change that coordinate system anymore.