ATMS Obs Validation

[ADCollard]

This issue will be used to track progress on the development and validation of radiances in UFO for GDAS in four stages:

1. Validation with GeoVaLs from GSI:

• [x] HofX

• The initial validation showed a large HofX difference (no QC) between GSI and JEDI for surface sensitive channels over snow area.

• CRTM has known issues with ATMS surface snow and ice emissivity models. CRTM has a float-point error when the input observations (surface channels) contain missing values. The issues and short-term fixes were documented in the following GitHub issues and PRs: Snow-related Issue: JCSDA-internal/ufo-old#1100 PR: DA-internal/crtm-old#74 --- snow fix in NESDIS_ATMS_SnowEM_Module.f90 Ice-related Issue: JCSDA-internal/ufo#621 PR: JCSDA-internal/crtm#371 --- sea ice fix in NESDIS_ATMS_SeaICE_Module.f90

• Both GSI and GDASApp for JEDI use CRTM-2.3.0. However, GSI CRTM does not have both ice/snow emissivity fix, and the GDASApp for JEDI does not have the ice emissivity fix. To have a fair comparison, CRTM in GSI and GDASApp has been updated with the fixes. The updated input obs and geoval files are available in the following location on ORION.

          Date: 2021080100
          Resolution: C768
          GeoDir=/work2/noaa/da/eliu/ufoeval/GSIobserver/2021080100/ioda/geovals/
          ObsDir=/work2/noaa/da/eliu/ufoeval/GSIobserver/2021080100/ioda/obs/
          BCDir=/work2/noaa/da/cmartin/UFO_eval/data/gsi_geovals_l127/nofgat_aug2021/20221027/bc/
          FixDir=/work2/noaa/da/cmartin/GDASApp/fix/

• With the updated CRTM, the HofX comparison (no QC) between GSI and UFO shows 100% agreement

  • [x] ATMS NPP
  • [x] ATMS N20

• [x] Update/improve QC filters (GSI QC steps in GSI setuprad.f90 and qcmod.f90 (qc_atms subroutine)

• [x] The @ObsError is set to missing in the input obs file
• [x] Code change is required for ObsErrorFactorSituDependMW to avoid getting the original obs error from the input obs file
• [x] Code change is required for ObsErrorFactorSituDependMW to fix a big - there should be no inflation based on the scattering index for ATMS.
• [x] YAML update
  • Improve the use of generic filters and obs functions in the YAML
  • Add filters to output diagnostic variables in YAML
  • Remove the need to repeatedly re-compute the same obs functions used in various filters
    • [x] QC count and obs error checks using-GSI produced GeoVaLs
• [ ] Validation Results: ObsError and HofX
  • [x] atms_npp - 100% replication except for one observation for channel 7 in the hydrometer check
  • [x] atms_n20 - 100% replication except for two observations for channel 7 in the hydrometer check Note: The differences likely come from numerical round-off errors.

2. UFO development:

• [ ] Separate HydrometeorCheckATMS into three separate checks for clarity --- work in progress
• [ ] Modify existing Gaussian thinning filter to reproduce GSI result --- work in progress
• [ ] Check with JEDI team about keeping the thinned data in the obs space
• [ ] Add thinning filter and a few more QC steps (from GSI read_bufrtovs.f90 routine) in YAML
• [ ] Validation with GSI result

3. IODA-converter development:

• [ ] Prepare ATMS YAML for bufr converter with channel remapping capability
• [ ] Work with Cory to add the capability of converting ATMS bufr to IODA in GDASApp workflow

4. HofX computation in FV3-JEDI with GDASApp workflow:

• [ ] Test workflow (plumbing)
• [ ] Check statistics
• [ ] Summarize and explain the differences between GSI and UFO

[ADCollard]

This also references #39

[emilyhcliu]

ATMS NPP (No QC) Surface-sensitive Channels channel-2	channel-16

channel-2	channel-16

Stratospheric Temperature Channels channel-11	channel-15

channel-11	channel-15

Moisture Channels (Tropospheric) channel-17	channel-22

channel-17	channel-22

Note: ATMS N20 also showed the similar large difference between GSI and UFO over snow area.

[emilyhcliu]

ATMS NPP (No QC) channel-1	channel-2

channel-3	channel-4

channel-5	channel-6

channel-7	channel-8

channel-9	channel-10

channel-11	channel-12

channel-13	channel-14

channel-15	channel-16

channel-1 7	channel-18

channel-19	channel-20

channel-21	channel-22

[emilyhcliu]

ATMS N20 (No QC) channel-1	channel-2

channel-3	channel-4

channel-5	channel-6

channel-7	channel-8

channel-9	channel-10

channel-11	channel-12

channel-13	channel-14

channel-15	channel-16

channel-1 7	channel-18

channel-19	channel-20

channel-21	channel-22

[emilyhcliu]

Initial (Original) / Final Observation Error Comparison: GSI vs. JEDI UFO

Result of QC filtering check (using ATMS N20 as an example) The following figures show the GSI final observation error on the left and UFO on the right The data count passed QC is listed above the color bar

Original Observation Error Channel-2 GSI	UFO

Observation Error after a series of data filtering and error inflation process: Channel-2 GSI	UFO

Channel-1 GSI	UFO

Channel-3 GSI	UFO

Channel-4 GSI	UFO

Channel-5 GSI	UFO

Channel-6 GSI	UFO

Channel-7 --- Difference in two observations GSI	UFO

Channel-8 GSI	UFO

Channel-9 GSI	UFO

Channel-10 GSI	UFO

Channel-11 GSI	UFO

Channel-12 GSI	UFO

Channel-13 GSI	UFO

Channel-14 GSI	UFO

Channel-15 GSI	UFO

Channel-16 GSI	UFO

Channel-17 GSI	UFO

Channel-18 GSI	UFO

Channel-19 GSI	UFO

Channel-20 GSI	UFO

Channel-21 GSI	UFO

Channel-22 GSI	UFO

[emilyhcliu]

HofX Comparison between GSI vs. JEDI UFO: No QC (Black) and QC (Red)

AMSU-N20

channel-1	channel-2

channel-3	channel-4

channel-5	channel-6

channel-7	channel-8

channel-9	channel-10

channel-11	channel-12

channel-13	channel-14

channel-15	channel-16

channel-17	channel-18

channel-19	channel-20

channel-21	channel-22

[emilyhcliu]

ATMS N20 Channel-7

Compared to GSI, UFO has two more points (south pole, sea ice) filtered out by HydrometeorCheck. (cloud/rain/precipitation screening)

GSI	UFO

latitude	longitude	ObsValue	Innovation	GsiEffectiveQC	EffectiveQC	EffectiveError
-73.179237	277.554077	207.757874	-0.062546	0	12	-3.368795e+38
-69.398537	359.459717	209.842499	0.037689	0	12	-3.368795e+38

ATMS NPP Channel-7

Compared to GSI, UFO has one more point (south pole, sea ice) filtered out by HydrometeorCheck. (cloud/rain/precipitation screening)

GSI	UFO

latitude	longitude	ObsValue	Innovation	GsiEffectiveQC	EffectiveQC	EffectiveError
-77.259377	172.936035	213.148590	-0.248276	0	12	-3.368795e+38