NOAA-EMC / JEDI-T2O

JEDI Transition to Operations activities.
GNU Lesser General Public License v2.1
5 stars 4 forks source link

MHS Obs Validation #45

Open ADCollard opened 1 year ago

ADCollard commented 1 year ago

Need to validate observations from MHS and evaluate QC procedures in the MHS YAML file, as a MHS YAML file already exists in the GDASApp.

This issue will be used to track progress on validation of radiances in UFO for GDAS in four stages:

ADCollard commented 1 year ago

This is the missing ifail_fact1 QC check:

  if(sea .or. ice .or. snow)then
     dsi=9.0_r_kind
     if(tb_obs(2) < h300)then
        dsi=0.13_r_kind*(tbc(1)-33.58_r_kind*tbc(2)/(h300-tb_obs(2)))
!       QC3 in statsrad
        if(luse .and. dsi >= one)aivals(10,is) = aivals(10,is) + one
     end if
!    si=42.72_r_kind+0.85_r_kind*tbc(1)-tbc(2)
  else
     dsi=0.85_r_kind*tbc(1)-tbc(2)
!    si=42.72_r_kind+0.85_r_kind*tb_obs(1)-tb_obs(2)
!    QC4 in statsrad
     if(luse .and. dsi >= one)aivals(11,is) = aivals(11,is) + one
  end if
  dsi=max(zero,dsi)
  fact1=((tbc(1)-7.5_r_kind*dsi)/r10)**2+(dsi)**2

  if(fact1 > one)then
     vfact=zero
!    QC1 in statsrad
     if(luse)aivals(8,is) = aivals(8,is) + one
     do i=1,nchanl
        if(id_qc(i) == igood_qc)id_qc(i)=ifail_fact1_qc
     end do
  else
     if (mhs) then  ! wv sounding channels
        do i=3,nchanl
           if (abs(tbc(i)) >= two) then
              varinv(i) = zero
              if(id_qc(i) == igood_qc)id_qc(i)=ifail_gross_routine_qc
           end if
        end do
     end if
     efact = (one-fact1*fact1)*efact
     vfact = (one-fact1*fact1)*vfact

I don't think we can do this in YAML. So it will require new code and ctests!

Or we can see what the exisiting Bennartz Scattering Test does.

azadeh-gh commented 1 year ago

image

azadeh-gh commented 1 year ago

We did not inflate error using mhs scattering index (similar to GSI) , but used Bennartz Scattering QC check in JEDI YAML file. So the obs errors are different between JEDI and GSI.

azadeh-gh commented 1 year ago

mhs_metob_b GSI vs JEDI HofX

gsi_hofx_vs_jedi_hofx_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature1

gsi_hofx_vs_jedi_hofx_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature2

gsi_hofx_vs_jedi_hofx_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature3

gsi_hofx_vs_jedi_hofx_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature4

gsi_hofx_vs_jedi_hofx_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature5

HofX_Difference_mhs_metop-b_diag_2021080100_mhs_metop-b

azadeh-gh commented 1 year ago

QC count check

gsi_hofx_vs_obs_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature1

gsi_hofx_vs_obs_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature2

gsi_hofx_vs_obs_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature3 gsi_hofx_vs_obs_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature4

gsi_hofx_vs_obs_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature5

PassedQC_Counts_Usedmhs_metop-b_diag_2021080100_mhs_metop-b QCCount_Diff_Used_mhs_metop-b_diag_2021080100_mhs_metop-b

azadeh-gh commented 1 year ago

EffectiveQC_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature1

GsiEffectiveQC_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature1

PreQC_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature1

EffectiveQC_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature2

GsiEffectiveQC_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature2

PreQC_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature2

EffectiveQC_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature3

GsiEffectiveQC_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature3

PreQC_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature3

azadeh-gh commented 1 year ago

OmF

OmFdiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature1 OmFdiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature2 OmFdiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature3 OmFdiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature4 OmFdiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature5 gsi_omb_vs_jedi_omb_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature1

gsi_omb_vs_jedi_omb_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature2 gsi_omb_vs_jedi_omb_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature3 gsi_omb_vs_jedi_omb_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature4 gsi_omb_vs_jedi_omb_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature5

azadeh-gh commented 1 year ago

Obs error check

errordiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature1

errordiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature2 errordiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature3 errordiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature4

errordiff_histogram_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature5

ObsErr_Difference_mhs_metop-b_diag_2021080100_mhs_metop-b

GsiFinalObsError_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature1 GsiFinalObsError_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature2

GsiFinalObsError_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature3 GsiFinalObsError_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature4

GsiFinalObsError_map_mhs_metop-b_diag_2021080100_mhs_metop-b_brightnessTemperature5

azadeh-gh commented 1 year ago

mhs_n19 JEDI vs GSI HofX were identical

gsi_hofx_vs_jedi_hofx_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature1

HofX_Difference_mhs_n19_diag_2021080100_mhs_n19

azadeh-gh commented 1 year ago

QC checks

PassedQC_Counts_Usedmhs_n19_diag_2021080100_mhs_n19

QCCount_Diff_Used_mhs_n19_diag_2021080100_mhs_n19

EffectiveQC_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature1 GsiEffectiveQC_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature1 PreQC_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature1 EffectiveQC_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature5 GsiEffectiveQC_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature5 PreQC_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature5

azadeh-gh commented 1 year ago

Obs Error

GsiFinalObsError_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature1 GsiFinalObsError_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature2 GsiFinalObsError_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature3 GsiFinalObsError_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature4 GsiFinalObsError_map_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature5

ObsErr_Difference_mhs_n19_diag_2021080100_mhs_n19 errordiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature1

errordiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature2 errordiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature3 errordiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature4 errordiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature5

azadeh-gh commented 1 year ago

OmF difference

OmFdiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature1

OmFdiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature2

OmFdiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature3 OmFdiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature4 OmFdiff_histogram_mhs_n19_diag_2021080100_mhs_n19_brightnessTemperature5