R2D2_v3 Adaptation and (Initial) Observation Database Development

[Dooruk]

This issue will evolve over time and there will be new steps/issues introduced as we continue developing a strategy.

I created these following initial discussions with @tariqhamzeyssai and @rtodling, please feel free to add and/or suggest edits any time. Most of these tasks will involve other GMAO folks.

Currently, all atmospheric observations are organized in PT6H windows.

Immediate needs:

• Indexing geos_atmosphere and geos_ocean IODAv3 observations that Swell already utilizes ( dated 20211212T000000Z, 20211212T060000Z) within the new R2D2_v3 API context and testing the search functionality.
• Implementing the new R2D2_v3 API within Swell

Short-term needs:

• Define an atmosphere "golden period" which ideally consists of two seasons. (Summer 2023, and ?) and identify the new observation sources and additional filtering needs (obs/UFO team involvement)
• Some obs yamls might require fewer changes (CRIS) whereas some of them need to be created from "scratch" (OMPS).
• Implement R2D2 indexing for a few days and test DA cycling

Long-term needs/thoughts:

• Current GEOS FP dataset consists of bufr and nc files which are not in IODAv3 format, this conversions has to take place.
• Devise a BUFR-to-IODAv3 conversion approach, employ some of the existing EMC methods (?), multiple options here:
  • The backend approach within JEDI (by defining the engine: type)
  • On-the fly conversion
  • Offline conversion (will create file duplicates)
• Processing bufr files via GEOS backgrounds
• How to ensure GSI filters (thinning) are maintained while we use the backend conversions.
• Devise a nc-to-IODAv3 conversion approach (same as above, could be done online or offline)
• Some of these conversions might split observations into multiple files, will that create a problem (metop/mtiasi) ?
• Identify additional ioda-conv needs for different types of observation types, for both atmosphere and ocean (e.g., mls_nrt converter is not available).
• Observation files version information (?)
• VarBC handling via R2D2 (?)
• (Ideally before the year 2100) Full GEOS FP (multi-month, multi-year) dataset

FYI @rgelaro

[rtodling]

Doruk, thanks for these items. I would remove the stuff about geovals. When dealing w/ the real (raw) obs, geovals is non-existent - it doesn't matter how longer it takes (it simply N/A).

[Dooruk]

Doruk, thanks for these items. I would remove the stuff about geovals. When dealing w/ the real (raw) obs, geovals is non-existent - it doesn't matter how longer it takes (it simply N/A).

Thanks, made the change.