search

jdkloe / pybufr-ecmwf

a python module that allows reading and writing BUFR formatted files, where BUFR stands for Binary Universal Form for the Representation of meteorological data.
Other
29 stars 12 forks source link

Error in descr = self.table_b[int_descr] to ECCODES ERROR : unable to get descriptor Y from table #10

Closed Akramz closed 7 years ago

Akramz commented 7 years ago

Hello,

I wanted to dump the content of a BUFR file using the file provided in the library "_bufr_toascii.py", and i got the following error : 

Traceback (most recent call last):
  File "org_bufr_to_csv.py", line 571, in <module>
    main()
  File "org_bufr_to_csv.py", line 540, in main
    separator, max_msg_nr, expand_flags)
  File "org_bufr_to_csv.py", line 58, in print_bufr_content1
    for msg_nr, msg in enumerate(bufr):
  File "/usr/local/lib/python2.7/dist-packages/pybufr_ecmwf/bufr.py", line 399, in messages
    self.get_next_msg()
  File "/usr/local/lib/python2.7/dist-packages/pybufr_ecmwf/bufr.py", line 378, in get_next_msg
    self.table_d_to_use, self.tables_dir)
  File "/usr/local/lib/python2.7/dist-packages/pybufr_ecmwf/bufr.py", line 91, in __init__
    self._bufr_obj.decode_data()
  File "/usr/local/lib/python2.7/dist-packages/pybufr_ecmwf/bufr_interface_ecmwf.py", line 964, in decode_data
    self.expand_raw_descriptor_list()
  File "/usr/local/lib/python2.7/dist-packages/pybufr_ecmwf/bufr_interface_ecmwf.py", line 1651, in expand_raw_descriptor_list
    self.bt.expand_descriptor_list(self.py_unexp_descr_list)
  File "/usr/local/lib/python2.7/dist-packages/pybufr_ecmwf/bufr_table.py", line 851, in expand_descriptor_list
    normalised_descriptor_list = self.normalise_descriptor_list(descr_list)
  File "/usr/local/lib/python2.7/dist-packages/pybufr_ecmwf/bufr_table.py", line 821, in normalise_descriptor_list
    descr = self.table_b[int_descr]
KeyError: 40239

After getting this error i thought i should try installing and dumping The BUFR using the ECCODES Library directly using the bufr_dump command, but then again i got a similar error : 

{ "messages" : [ 
ECCODES ERROR   :  unable to get descriptor 040239 from table
ECCODES ERROR   :  unable to get descriptor 040232 from table
ECCODES ERROR   :  unable to get descriptor 040237 from table
ECCODES ERROR   :  unable to get descriptor 040231 from table
ECCODES ERROR   :  unable to get descriptor 040233 from table
ECCODES ERROR   :  unable to get descriptor 040235 from table
ECCODES ERROR   :  unable to get descriptor 040236 from table
ECCODES ERROR   :  unable to get descriptor 040197 from table
ECCODES ERROR   :  unable to get descriptor 040240 from table
ECCODES ERROR   :  unable to get descriptor 040220 from table
ECCODES ERROR   :  unable to get descriptor 040221 from table
ECCODES ERROR   :  unable to get descriptor 040223 from table
ECCODES ERROR   :  unable to get descriptor 010220 from table
ERROR: unable to unpack data section: Function not yet implemented (message=1)

I guess the error is related to my file but i know how, i am new to BUFR and i don't know how to read or debug related errors, any hint or word of advice would be much appreciated.

Thank you.

jdkloe commented 7 years ago

Hi Akramz,

the BUFR file format is a binary format, and to decode it a table with detailed definition of its content is needed (i.e. which variable takes how many bits in the file, how to scale it, etc.) This definition usually is stored in so called BUFR tables, and I suspect there is your problem. WMO defines standard tables, a new version each year, and it seems you have a file that cannot be decoded using these standard tables. This results in a keyerror during decoding. In addition some BUFR files have been created using custom definitions, which may be independent from the definitions provided by WMO. The key code in the error message confirms this. It ends with 239, and all keys ending with a number between 192 and 255 are reserved for local use, so not provided by WMO. Therefore you need to have a copy of the BUFR table files that where used to create this BUFR file, only then you can decode it. You should contact the person or institute that generated this BUFR file and ask them to provide such BUFR table files. One issue you may encounter, is the fact that the format of BUFR table files is not defined in any standard, so different types of BUFR endocing/decoding software use different formats for the BUFR table files. If needed I can assist you to convert them to the right format needed by pybufr-ecmwf. Once you have these table files feel free to contact me again.

jdkloe commented 7 years ago

marked the issue as enhancement, since the error message could be made a lot clearer.

jdkloe commented 7 years ago

Additional explanation was added to the error message with commit #484. Closing this request.