Observer End-to-End Test for GOES ABI AMVs

[emilyhcliu]

This issue is to document the end-to-end tests for GOES-AMVs

Observer End-to-End Tests for GOES-AMVs processing

• [x] Develop BUFR converter application for satwind - python script and json configuration
• [x] Test the observer process from IODA to H(x) for GOES-17 and GOES-16
  • Verification of HofX, O-F, Obs Error, and QC filtering with GSI
• [x] Test the observer process from IODA to H(x) + initial error assignment + read routine QC (Basic QC - obs tossed but not passed to setup routine)
• [x] Test BUFR converted IODA obs input to H(x) + initial error assignment + read routine QC + setup routine QC

This is a plumbing test between IODA and UFO. Scientific assessment will be performed later by AMV experts.

Check results for GOES-16 and GOES-17

[emilyhcliu]

Look at Obs (IODA obs converted from BUFR dump file) - HofX

GOES-17 AMVs windEastward (U-wind) O-F

• 6-hour dump file - contains 6 hourly data frames (6 unique time stamps)
• AMV ObsType: 240 (IRSW), 245 (IRLW), 246 (WVCT), 247 (WVCA/DL), 251 (VIS)

1st Hour (Timeframe 1)	2nd Hour (Timeframe 2)

3rd Hour (Timeframe 3)	4th Hour ITimeframe 4)

5th Hour (Timeframe 5)	6th Hour (Timeframe 6)

Each Timeframe contains AMVs retrieved from 5 algorithms (240, 245, 246, 246, 251) Example: Timeframe 3

Type 240 (IRSW)	Type 245(IRLW)

Type 246 (WVCT)	Type 247(WVCA/DL)

Type 251 (VIS)	All Types

[emilyhcliu]

Compare GOES-17 AMV windEastward O-F between UFO and GSI (1st Hour)

UFO O-F without QC

UFO O-F with Basic QC- exclude data with the following conditions: (1) satellite zenith angle > 68 (2) obs pressure > 125 mb (3) VIS data (ObsType 251)

UFO O-F with Basic QC- exclude data with the following conditions: (1) saellite zenith angle > 68 (2) obs pressure > 125 mb (3) VIS data (ObsType 251) (4) data over non-water surface + latitude > 20 for Types 240 and 245 These QC steps are GSI read routine QC --- obs tossed without passing to setup routine

Break down for data passed read-routine QC Type 245	Type 246	Type 247

GSI O-F (with read routine QC; no setup routine QC applied)

UFO O-F with QC:

GSI O-F with QC:

[emilyhcliu]

Compare GOES-16 AMV windEastward O-F between UFO and GSI (1st Hour)

O-F without QC

O-F with Basic QC- exclude data with the following conditions: (1) satellite zenith angle > 68 (2) obs pressure <125 mb (3) VIS data (ObsType 251)

O-F with Basic QC- exclude data with the following conditions: (1) satellite zenith angle > 68 (2) obs pressure <125 mb (3) VIS data (ObsType 251) (4) data over non-water surface + latitude > 20 for Types 240 and 245 These QC steps are GSI read routine QC --- obs tossed without passing to setup routine

Break down data pass GSI read routine QC

Type 240	Type 245

Type 246	Type 247

GSI O-F (with read routine QC; no setup routine QC applied)

UFO O-F with QC:

GSI O-F with QC:

[emilyhcliu]

Notes - general:

• need to replace satelliteZenithAngle with sensorZenithAngle
• need to check the use of land_type_index_NPOESS in the filters. land_type_index _NPOESS (ranges from 0 to 19) is not the typical land/sea/sea-ice/snow index (0, 1,2, 3) For now, I replace the land_type_index_NPOSS with water_area_fraction/land_area_fraction) in the filters.

Notes - specific:

• ObsErrorFactorPressureCheck had been updated (by GMAO). Two more parameters are required surface_obs and an error inflation variable. These additional required parameters break our ObserrorFactorPressureCheck --- need to investigate
• ObsErrorFactorDuplicateCheck requires PreUseFlag in the obs file. The IODA obs file converted from BUFR does not have this variable. We should revisit this filter.

In the GOES AMV QC test: ObsErrorFactorPressureCheck and ObsErrorFactorDuplicateCheck are turned off In addition, gross check is also turned off since this check depends on error inflation from ObsErrorFactorPressureCheck.

[BrettHoover-NOAA]

land_type_index _NPOESS (ranges from 0 to 19) is not the typical land/sea/sea-ice/snow index (0, 1,2, 3) For now, I replace the land_type_index_NPOSS with water_area_fraction/land_area_fraction) in the filters.

Hi @emilyhcliu, I was not aware that land_type_index_NPOESS and the GSI's surface type variable are not intended to be identical. This affects QC of both satellite winds and scatterometer winds since we are using the NPOESS variable in place of the GSI's variable for QC based on surface type. There is one particular case with scatterometer winds where the UFO and GSI disagree on QC because they have different surface type categories, but your description of the NPOESS variable here makes me think that the disagreement on surface type isn't straightforward. Do you have a reference for what the 20 categories are for the NPOESS variable, in case I need to update the satwind and scatwind YAMLs?

[emilyhcliu]

@BrettHoover-NOAA The definition of NPOESS land index type can be found in crtm_interface.f90

  ! Indices for the CRTM NPOESS EmisCoeff file
  integer(i_kind), parameter :: INVALID_LAND = 0
  integer(i_kind), parameter :: COMPACTED_SOIL = 1
  integer(i_kind), parameter :: TILLED_SOIL = 2
  integer(i_kind), parameter :: IRRIGATED_LOW_VEGETATION = 5
  integer(i_kind), parameter :: MEADOW_GRASS = 6
  integer(i_kind), parameter :: SCRUB = 7
  integer(i_kind), parameter :: BROADLEAF_FOREST = 8
  integer(i_kind), parameter :: PINE_FOREST = 9
  integer(i_kind), parameter :: TUNDRA = 10
  integer(i_kind), parameter :: GRASS_SOIL = 11
  integer(i_kind), parameter :: BROADLEAF_PINE_FOREST = 12
  integer(i_kind), parameter :: GRASS_SCRUB = 13
  integer(i_kind), parameter :: URBAN_CONCRETE = 15
  integer(i_kind), parameter :: BROADLEAF_BRUSH = 17
  integer(i_kind), parameter :: WET_SOIL = 18
  integer(i_kind), parameter :: SCRUB_SOIL = 19

[BrettHoover-NOAA]

Thank you!

[emilyhcliu]

The calculation of tropopause pressure is not correct. The bug fix had been checked into fv3-jedi repository. See details in the following PRs: GDASApp PR #575 fv3-jedi PR #1001 More Informationon Tropopause Pressure Fix

[emilyhcliu]

Notes: Check GeoVaLs: slmsk and area_fraction (water, ice, snow, and land)

Model: slmsk: 0, 1, 2 (sea, land, ice) GeoVaLs: slmsk: 0, 1, 2, 3, 4, 5 (model slmsk + 3 ---> 0+3, 1+3, 2+3 ---> 3, 4, 5

In GSI: for satwind use idomsfc from deter_sfc2 use iflg from deter_sfc_type

• idomsfc is determined by ilsi_full and weights (dist between grid points and obs)

  idomsfc is determined by model slmsk and slmsk+3 idomsfc ranges from 0, 1, 2, 3, 4, 5

• isflg is deermined by islis_full and weights ---> sfcpct (surface type %) ---> isflg

  surface type % > 0.99 ---> isflg - surface_type isflg = 0 (sea); 1 (land); 2 (ice); 3 (snow); 4 (mixed)

[emilyhcliu]

I replaced the use of water_area_fraction and land_area_fraction with slmsk This changed the results slightly but still very close to the GSI results.

The slmsk is not a good ideal.
Go back to use water_area _fraction and land_area_fraction.

[emilyhcliu]

Correct a bug in using the threshold for water_area fraction. The results between GSI and UFO are closer with the bug fix.

[emilyhcliu]

GOES-17 Final OMF GOES-17 Final Obs Error GOES-16 Final OMF GOES-16 Final Obs Error