Observer End-to-End Test for OMPS TC

[emilyhcliu]

End-to-End Tests for OMPS TC processing

• [x] Create BUFR converter - python script
• [x] Test BUFR converted IODA obs input to H(x)
• [x] Test BUFR converted IODA obs input to H(x) + initial error assignment + read routine QC
• [x] Test BUFR converted IODA obs input to H(x) + initial error assignment + read routine QC + setup routine QC

Issues found in the existing data filtering yaml (omi_aura.yaml) are documented here:

• No observation error assignment (all filled with missing values)
• Scan position + latitude check is not accurate
• Gross check was not implemented correctly
• Quality control variable types do not comply with IODA convention --- they need to be integers
• Quality control variable names do not comply with IODA convention --- should be camel case

Updates made and tested

• Added observation error assignment and fixed scan position/latitude and gross check
• The quality control flags (toqc, toqf, and afbo) are integers. Modified the filters accordingly in the YAML
• The quality control flags names do not comply IODA convention (should be camel case). TOQC - total ozone error flag (0, 1 are good) ---> totalOzoneQualityCode AFBO - algorithm flag ( exclude 3, 13 indicating data retrieved with c-pair algorithm using 331 and 360 nm) ---> bestOzoneAlgorithmFlag

---

QC Procedures: (Data thinning not applied)

read routine QC

• Observation value sanity check
• Total Ozone Quality Code Check (TOQC) --- accept TOQC = 0, 1 only
• Algorithm Check --- remove data retrieved using the c-pair algorithm (3, 13)

setup routine QC

• Scan position + latitude check
• Background departure gross check |O-F| > min(10*obserr, 300)

[emilyhcliu]

UFO HofX without QC

UFO HofX with QC

GSI HofX with QC