search

NOAA-EMC / JEDI-T2O

JEDI Transition to Operations activities.
GNU Lesser General Public License v2.1
5 stars 4 forks source link

Evaluate specific humidity QC Procedures in Aircraft YAML File and GSI #49

Open nicholasesposito opened 1 year ago

nicholasesposito commented 1 year ago

An aircraft YAML file exists in GDASApp copied from UFO/ewok/jedi-gdas on June 28, 2022.

Before diving into replicating the PrepBUFR preprocessing for aircraft observations, we should first check that the QC procedures implemented in UFO match those being performed in GSI on PrepBUFR aircraft observations.

[] Run GDASApp for first week of August 2021 for aircraft observations [] Compare EffectiveQC against GsiEffectiveQC for specific_humidity [] If there are differences, look at setupq.f90 in the GSI source code. [] Make any modifications to YAML as necessary

This issue will be used to track progress, including figures, statistics, and paths to files.

nicholasesposito commented 1 year ago

I initially went through setupq.f90 in the GSI code to determine what QC was done to specific_humidity for aircraft data. There was none, so I took out all the QC in the yaml, ran it and produced the following results:

The GSI HofX and Jedi HofX line up perfectly (below) , as does GSI omb vs Jedi omb (not pictured but looks the same) gsi_hofx_vs_jedi_hofx_2021080100_aircraft_specific_humidity

The hofxdiff histogram shows differences around 10e-8, which is to be expected. Errordiff was also around 0. hofxdiff_histogram_2021080100_aircraft_specific_humidity

The GSI HofX vs Observations plot, and the Jedi HofX vs Observations plot looked the exact same, which is great. gsi_hofx_vs_obs_2021080100_aircraft_specific_humidity

nicholasesposito commented 1 year ago

Finally I performed some statistics on the output file, and found that:

num of points where specific_humidity/EffectiveQC=0 : 8070 num of points where specific_humidity/GsiEffectiveQC=0 : 8070 total same = 8070 of 120387 percent same = 100 % num of points where effqc = 0 and gsieffqc = 1 is: 0 num of points where effqc = gsieffqc is: 8070 ( only happened where 0 = 0)

num of points where effqc >= 1 and gsieffqc >= 1 is: 4263 It should be noted that for the last number, the likely values could have been 1, 10, or 11. The PreQC values were all over the place. 9 was the most common (3885 of 4263), followed by 14 ( 188), 13 (167), and 15 (22)