NOAA-EMC / GDASApp

Global Data Assimilation System Application
GNU Lesser General Public License v2.1
15 stars 31 forks source link

[BUG] BUFR2IODA conversion problem for altkob data type due to missing associated json file #877

Open emilyhcliu opened 8 months ago

emilyhcliu commented 8 months ago

Run BUFR2IODA process with the latest Gloibal Workflow and GDASApp (uses submodules) One obs "altkobs" failed to process correctly.

The error message is:

gen_bufr2ioda_json.py: Using /work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/parm/ioda/bufr2ioda/bufr2ioda_altkob_surface.json as input

  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/ush/run_bufr2ioda.py", line 119, in <module>
    bufr2ioda(args.current_cycle, args.RUN, args.DMPDIR, args.config_template_dir, args.COM_OBS)
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/ush/python/wxflow/logger.py", line 266, in wrapper
    retval = func(*args, **kwargs)
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/ush/run_bufr2ioda.py", line 70, in bufr2ioda
    gen_bufr_json(config, template, json_output_file)
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/gen_bufr2ioda_json.py", line 19, in gen_bufr_json
    bufr_config = parse_j2yaml(template, config)
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/ush/python/wxflow/yaml_file.py", line 176, in parse_j2yaml
    yaml_dict = YAMLFile(data=yaml_file)
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/ush/python/wxflow/yaml_file.py", line 35, in __init__
    self.update(config)
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/ush/python/wxflow/attrdict.py", line 120, in update
    other.update(args[0])
ValueError: dictionary update sequence element #0 has length 1; 2 is required
+ JGLOBAL_ATM_PREP_IODA_OBS[1]: postamble JGLOBAL_ATM_PREP_IODA_OBS 1706204081 1

This error message is caused by missing json file for altkobs The gen_bufr2ioda_json.py can not find the json for alkobs because it is missing

Orion-login-3[121] eliu$ ls -l /work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/parm/ioda/bufr2ioda/bufr2ioda_altkob_surface.json
ls: cannot access /work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/parm/ioda/bufr2ioda/bufr2ioda_altkob_surface.json: No such file or directory

NOTE: I found another one trackob failed to process correctly, not because of missing the associated json file. Under investigating...

OK. Found a few data types failed....

Traceback (most recent call last):
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_bathythermal_profiles.py", line 256, in <module>
    log_level = 'DEBUG' if args.verbose else 'INFO'
NameError: name 'args' is not defined

Traceback (most recent call last):
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_tesac_mammals_profiles.py", line 301, in <module>
    log_level = 'DEBUG' if args.verbose else 'INFO'
NameError: name 'args' is not defined

Traceback (most recent call last):
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_xbtctd_profiles.py", line 282, in <module>
    log_level = 'DEBUG' if args.verbose else 'INFO'
NameError: name 'args' is not defined

Traceback (most recent call last):
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_tesac_profiles.py", line 301, in <module>
    log_level = 'DEBUG' if args.verbose else 'INFO'
NameError: name 'args' is not defined

Traceback (most recent call last):
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_subpfl_glider_profiles.py", line 307, in <module>
    log_level = 'DEBUG' if args.verbose else 'INFO'
NameError: name 'args' is not defined

Traceback (most recent call last):
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_subpfl_argo_profiles.py", line 306, in <module>
    log_level = 'DEBUG' if args.verbose else 'INFO'
NameError: name 'args' is not defined

These are marine data. The input argument parameter for log_level is not defined in the routine listed above.

ShastriPaturi commented 8 months ago

@emilyhcliu, my bad. I did not add the json file.

emilyhcliu commented 8 months ago

@ShastriPaturi. Just one question. Are you processing the BUFR from the tank or dump?

ShastriPaturi commented 8 months ago

@emilyhcliu, the dump

emilyhcliu commented 8 months ago

@emilyhcliu, the dump

Got it. Thanks.

ShastriPaturi commented 8 months ago

@emilyhcliu please see PR #879