Fearture/jjin3/allow for missing ob type b

[gmao-jjin3]

Allow filter for missing satellite radiance files in the final SWELL configurations, following @rtodling PR https://github.com/GEOS-ESM/swell/pull/400. However, radiance observations with zero-sized input files are omitted in the final configurations because JEDI/IODA cannot read those zero-sized files. In addition, filters for missing conventional and ozone observational files are omitted since an action of "error" is now assigned for those missing observations.

[gmao-jjin3]

So far, this PR works for missing convectional, ozone, and microwave radiance data. It crashes swell-3dfgat_atmos run when any of AIRS, IASI, CRIS-FSR, and AVHRR input files is missing.

[gmao-jjin3]

@gmao-wgu It turns out that JEDI crashes with this correlated IR observational error inflation filter when there aren't observational input files:

- filter: BlackList
  filter variables:
  - name: brightnessTemperature
    channels: *cris-fsr_n20_channels
  action:
    name: inflate error
    inflation variable:
      name: ObsFunction/ObsErrorFactorWavenumIR
      channels: *cris-fsr_n20_channels
      options:
        channels: *cris-fsr_n20_channels

UserError: Invalid prior filter requested ObsErrorCrossVarCov: Obs error correlations not provided for 324 of 431 channels/variables in file: fv3-jedi/rcov/cris-fsr_108_jedi_rcov.nc4. To see which channels, turn on OOPS_TRACE OOPS Ending 2024-09-03 12:25:59 (UTC-0400)

I added a check of number of observations in /discover/nobackup/jjin3/jedi/jedi-work2/ufo/src/ufo/errors/ObsErrorCrossVarCov.cc.jj . However, that doesn't solve the issue. JEDI still returns an error of "Invalid prior filter". Any idea?

[gmao-wgu]

@gmao-jjin3 I will try to replicate your case and proceed from there.

[gmao-jjin3]

@gmao-wgu It turns out there is an issue to handle IASI files in JEDI/ioda. The filter mentioned above is fine. My other tests go through with missing AIRS and CRIS-FSR input observational files. Tests only fail with missing IASI data. UFO filters are fine. Reported the issue to the JCSDA team: https://github.com/JCSDA-internal/ioda/issues/1332

[gmao-jjin3]

It turns out I had a minor update in UFO before I traced down the above issue in IODA. Here is a UFO PR for the update: https://github.com/JCSDA-internal/ufo/pull/3455.

[Dooruk]

@gmao-jjin3 and @rtodling, is this ready to merge? I see the relevant PR in JEDI is merged.

[gmao-jjin3]

@Dooruk No, the JEDI team is still looking into the issue that arises with missing IASI data.

[fabiolrdiniz]

@gmao-jjin3, please notice that the actual error message in your excerpt above is:

UserError: Invalid prior filter requested

which means you are probably adding a filter in the wrong section (pre, prior, post, please see https://github.com/JCSDA-internal/oops/blob/d77217323bbbd02ea41049fe1bd339f24a64f177/src/oops/base/ObsFilters.h#L41-L58 for more info), while

ObsErrorCrossVarCov: Obs error correlations not provided for 324 of 431 channels/variables in file: fv3-jedi/rcov/cris-fsr_108_jedi_rcov.nc4. To see which channels, turn on OOPS_TRACE
OOPS Ending 2024-09-03 12:25:59 (UTC-0400)

is a warning message that the matrix you provided for CrIS-FSR doesn't have covariances for some channels (likely the ones you don't use).

[gmao-jjin3]

@fabiolrdiniz I noticed that before. However, I don't think it is the real reason. All the filters and their order are fine when the obs input file is available.