search

NCAR / ucomp-pipeline

Data processing pipeline for UCoMP
Other
6 stars 3 forks source link

Reprocess test dates before 0.5.x release/reprocessing #155

Closed mgalloy closed 11 months ago

mgalloy commented 1 year ago

Reprocess the following dates:

Check the following:

[!NOTE] Time estimate: This issue, as stated, is complete.

mgalloy commented 1 year ago

Reprocessing starting, using ucomp.workshop-test.cfg configuration file. Results will be in:

/hao/dawn/Data/UCoMP/process.workshop-text
mgalloy commented 1 year ago

20210719 is done. Most of the 1074, all of the 1079, and one 789 file pass GBU. Everything else fails.

bberkeyU commented 1 year ago

I agree with the GBU judgments on 789 and 1074, but I am not sure I agree with the reasons to kick them out of the 789. These two were rejected for the reason of 16: spurious V signal. But in the quick looks they just look like column noise, not excess V. This isn't a pipeline metric but I would by eye reject them for strong circular fringes.

Digging down a little more the 637 have some issues but they are also rejected for reason 16 but I dont see any excess V. It looks like the excess V really responds to faint images without great column corrections, ie when NUC=NORMAL.

mgalloy commented 1 year ago

20220325 and 20220523 are done now too.

mgalloy commented 1 year ago

The field radius of 700 seems good. It is marked on the below image of center line intensity in 20220712.171521.ucomp.637.l1.3.fts:

field-radius-700

mgalloy commented 1 year ago

20220712 finished last night.

jburkepile commented 1 year ago

20210719 reprocessed data comments

From what I have seen at first glance the reprocessing produced multiple positive results for this day

The 'culling' of bad pixels in the outer field in: Line width; Doppler velocity; linear polarization; and the two azimuths has definitely helped reduce, but not eliminate, the number of noisy pixels in the outer FOV. Some of these noisy pixels have values that are in the 'reasonable' range. We will need to identify additional 'tests' to eliminate these pixels after the workshop; perhaps based on the line profiles of the pixels above a certain radius.

The 789, and 1079 nm , doppler images look much better

The new reprocessing did not produce the 637 and 706 level 2 WHICH IS GOOD. Those 2 wavelengths were very poor. I don't know if they can be salvaged.

I am having to adjust to the new line width scaling. The images do appear quite different! I do like the new, larger annotation.