Inconsistency between bufr2ioda iodafile and init obsdatain

[RussTreadon-NOAA]

The following bufr2ioda converters add the string data_format to the name of the iodafile created by the converter

./bufr2ioda_sfcshp_prepbufr.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
./bufr2ioda_snocvr_bufr.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
./bufr2ioda_gpsro_bufr_combined.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
./bufr2ioda_acft_profiles_prepbufr.py:    iodafile = f"{cycle_type}.t{hh}z.acft_profiles.{data_format}.nc"
./marine/bufr2ioda_insitu_surface_altkob.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bufr2ioda_insitu_profile_marinemammal.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bufr2ioda_insitu_surface_trkob.py:    iodafile = f"{cycle_type}.t{hh}z.insitu_surface_{data_format}.{cycle_datetime}.nc4"
./marine/bufr2ioda_insitu_profile_bathy.py:    iodafile = f"{cycle_type}.t{hh}z.insitu_profile_{data_format}.{cycle_datetime}.nc4"
./marine/bufr2ioda_insitu_profile_xbtctd.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc4"
./marine/bufr2ioda_insitu_profile_tesac.py:    iodafile = f"{cycle_type}.t{hh}z.insitu_profile_{data_format}.{cycle_datetime}.nc4"
./marine/bkp/bufr2ioda_bathythermal_profiles.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bkp/bufr2ioda_trackob_surface.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bkp/bufr2ioda_subpfl_argo_profiles.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bkp/bufr2ioda_xbtctd_profiles.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bkp/bufr2ioda_altkob_surface.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bkp/bufr2ioda_tesac_mammals_profiles.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bkp/bufr2ioda_tesac_profiles.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bkp/bufr2ioda_subpfl_glider_profiles.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc"
./marine/bufr2ioda_insitu_profile_glider.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}_profiles.{data_format}.nc4"
./bufr2ioda_adpupa_prepbufr.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
./bufr2ioda_gnssro_bufr.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}.tm00.{data_format}.nc"
./bufr2ioda_adpsfc_prepbufr.py:    iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
./bufr2ioda_conventional_prepbufr_ps.py:    iodafile = f"{cycle_type}.t{hh}z.{data_description}.{data_format}.nc"

Inclusion of data_format in the ioda filename is at odds with the obsdatin filename written to the var and local ensemble yamls by the init jobs. The obsdatain filenames do not include the data_format string. This inconsistency causes the var and local ensemble init jobs to abort because an expected ioda observation file is not found.

For example, ./bufr2ioda_gnssro_bufr.py creates ioda file gdas.t18z.gnssro.bufr_d.nc. However, the var yaml expects to read gdas.t18z.gnssro.tm00.nc. The init jobs does not find this file and aborts

OSError: unable to copy /scratch1/NCEPDEV/stmp2/role.jedipara/COMROOT/prjedi/gdas.20240502/18/obs/gdas.t18z.gnssro.tm00.nc to /scratch1/NCEPDEV/stmp2/role.j\
edipara/RUNDIRS/prjedi/gdasatmanl_18/obs//gdas.t18z.gnssro.tm00.nc
+ JGLOBAL_ATM_ANALYSIS_INITIALIZE[1]: postamble JGLOBAL_ATM_ANALYSIS_INITIALIZE 1721392162 1

[RussTreadon-NOAA]

Also find that some ioda files include the string tm00 whereas other ioda files do not contain the string tm00. This causes problems in the init jobs. The templates currently assume atmosphere_obsdatain_suffix: ".tm00.nc"

[RussTreadon-NOAA]

The following ush/ioda/bufr2ioda scripts have been modified to make the output ioda filename consistent with JCB templates

ush/ioda/bufr2ioda/bufr2ioda_acft_profiles_prepbufr.py
ush/ioda/bufr2ioda/bufr2ioda_adpsfc_prepbufr.py
ush/ioda/bufr2ioda/bufr2ioda_adpupa.py
ush/ioda/bufr2ioda/bufr2ioda_adpupa_prepbufr.py
ush/ioda/bufr2ioda/bufr2ioda_conventional_prepbufr_ps.py
ush/ioda/bufr2ioda/bufr2ioda_gnssro_bufr.py
ush/ioda/bufr2ioda/bufr2ioda_gpsro_bufr_combined.py
ush/ioda/bufr2ioda/bufr2ioda_sfcshp_prepbufr.py
ush/ioda/bufr2ioda/bufr2ioda_snocvr_bufr.py

Converter scripts in ush/ioda/bufr2ioda/marine are intentionally left alone.

@emilyhcliu , shall I create a branch & open a PR to get these changes into GDASApp develop?

[RussTreadon-NOAA]

Work for this issue will be done in RussTreadon-NOAA:feature/bufr2ioda.

[emilyhcliu]

@RussTreadon-NOAA Please go ahead and open the PR. Thanks for taking care of these necessary changes.