error in decoding bufr (missing table?)

[brancomat]

The attached file SPC.zip contains 4 GTS bufr messages which cannot be parsed by wreport:

$ dbamsg scan --verbose IUSD01ERSA190000.BIN
could not detect the encoding of IUSD01ERSA190000.BIN

$ dbamsg scan -t bufr --verbose IUSD01ERSA190000.BIN 
Cannot parse BUFR message #0: BUFR table for center 80:0 table 0:29:0 not found at offset 20.
#0 BUFR message: 109724 bytes.

I labeled this as a question since I'm not sure if the bufr is malformed (their fault) or we're missing a table (our fault)

[brancomat]

A radiosonde technician claims that the new files refer to new wmo tables. This is becoming a compelling issue but I'm not particular tech savy in this topic and the table update it's kind of a gray zone betweend developer and users. (if there are reproducibile and documentable steps from the analysis of a bufr to the update of the bufr tables and someone is willing to teach me, I offer to write it down as a part of the wreport doc)

[brancomat]

A radiosonde technician claims that the new files refer to new wmo tables. This issue became kind of compelling so I'll mix a little assignees and labels. I'm not particularly tech savy on the topic and the table update is kind of a gray area between devel and end users, if there are reproducible steps from the bufr analisys to the table update and someone is willing to teach me I could write it down as an addendum to the wreport doc.

[dcesari]

As far as I can reconstruct, the table update procedure is done with wrep-importtable. By default it tries to download the URL http://www.wmo.int/pages/prog/www/WMOCodes/WMO306_vI2/LatestVERSION/latestTables.zip which is no more available on WMO site, so we should go to http://www.wmo.int/pages/prog/www/WMOCodes/WMO306_vI2/LatestVERSION/LatestVERSION.html download by hand -please correct me if I'm wrong- "Machine readable text and xml files (see Notes 4 and 5)" in the section "FM 94 BUFR and FM 95 CREX", which is at version-dependent URL http://www.wmo.int/pages/prog/www/WMOCodes/WMO306_vI2/LatestVERSION/BUFRCREX_31_0_0.zip then run

wrep-importtable BUFRCREX_31_0_0.zip

Unfortunately, after some work and the production of a new B table, it stops with the following error:

...
Writing B0000000000000031000.txt...
Writing B003100.txt...
Traceback (most recent call last):
  File "/usr/bin/wrep-importtable", line 272, in <module>
    tables.write_files()
  File "/usr/bin/wrep-importtable", line 251, in write_files
    self.table_b.write_files()
  File "/usr/bin/wrep-importtable", line 141, in write_files
    print(line, file=out1)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2030' in position 73: ordinal not in range(128)

Presumably some unicode characters have appeared in WMO tables. so this is becoming an open issue for wrep-importtable, or I can open a new one, if anybody can confirm that the procedure I followed was correct.

[spanezz]

I'll now try to import tables and fix the script accordingly

[spanezz]

I ported wrep-importtable to python3, removed the dowload feature, and imported all tables until the last one. Now the files in SRC.zip seem to be decoded fine.