Error: BUFRLIB: NEMTBD - REPLICATOR IS OUT OF ORDER IN TABLE D SEQUENCE MNEMONIC MSTTB001

[LucaFibbi]

When I decode some BUFR files received through the Global Telecommunication System of the WMO using this library (e.g. using the debufr program) I get the error:

*BUFR ARCHIVE LIBRARY ABORT** BUFRLIB: NEMTBD - REPLICATOR IS OUT OF ORDER IN TABLE D SEQUENCE MNEMONIC MSTTB001
*BUFR ARCHIVE LIBRARY ABORT**

The BUFR code tables used are those present in the tables directory of this repository. If the same BUFR files are decoded with some on-line decoders (e.g. https://kunden.dwd.de/bufrviewer/), the decoding takes place without any errors. I am attaching some examples of BUFR files with this problem. bufr.zip

[jbathegit]

Hello @LucaFibbi, and thank you for reporting this to us!

I took a look at the 3 messages in the bufr.zip file you attached, and it turns out the library is correct that there are errors in the encoding of the messages. Specifically, if you run debufr with the -b option, it will print out all of the BUFR descriptors in Section 3 without trying to actually decode the data values in Section 4. When you do this, you can see for yourself that in each of the messages there are occurrences of the descriptor '031001' in the Section 3 descriptor list.

The problem is that '031001' is used solely for storing a delayed replication count, so it's only valid when it immediately follows a delayed replication descriptor of the form '1XX000', and where XX is the number of descriptors to be replicated. So, bottom line, the library is doing the right thing by diagnosing this as an error, and I'm honestly stumped as to how the on-line decoders you referred to (or, for that matter, any proper BUFR decoder?) could overcome that particular error to make any sense of what's in the message.

Again, the descriptor '031001' is illegal in any context other than immediately following a '1XX000' descriptor, which you can see for yourself isn't present in any of these messages. We've actually already noticed this issue ourselves in some of the data we've received operationally at NCEP/EMC, and we've already separately contacted the folks at the Korea Meteorological Administration ('RKSL' is the originator for Seoul, South Korea) to report this to them and request that they fix it.

[LucaFibbi]

Hello Jeff,

thank you for the your answer. I apologize for the late in the my answer, but only now I had the time to do some tests that add more information to the problem. I agree with your considerations: the descriptor '031001' is illegal in any context other than immediately following a '1XX000' descriptor. The BUFR message shouldn't contain this type of descriptor without a precedent '1XX000' descriptor. But BUFR message with this type of problem are continuing to arrive from 'RKSL' originator centre and they seem to contain plausible meteorological data if decoded with other software. I have tested bufr_dump program of ecCodes package developed by ECMWF. I have attached the output of bufr_dump for the three examples of BUFR files with this problem that I already sended. I have used bufr_dump with the option -ja (output files bufr_dump_ja) and with the option -O (output files bufr_dump_O). I have sum the "width" of the descriptors of section 3 indicated in the files bufr_dump_ja and then I padding the result at 8 bits and I get exactly the size of bufr Data Encoded indicated in the files bufr_dump_O. Eg. the sum of width of the descriptors of message 1 subset 1 in the file ISSX01RKSL310600.BIN.bufr_dump-ja.out is 1030 bits that it's padded at 1032 and it's equal at bufrDataEncoded 129 bytes (1032/8=129). When decode the section 4 the bufr_dump program read a value of null (missing) for the '031001' descriptor and, for the absence of precedent '1XX000' descriptor or for the its null value, continue to read the value of the successive descriptor considering zero replications. The values of the descriptors before and after the '031001' descriptor seem to be plausible when they not have missing value. Is the bufr_dump behavior correct ? If the section 4 is correct only before the '031001' descriptor, is possible decode the BUFR until the '031001' descriptor and then print only a warning and continue with successive subset or BUFR message. I am available for any further tests or to make changes to the code or, if useful, further investigate how bufr_dump overcomes this particular error in BUFR message. bufr_dump_output.zip

Hello @LucaFibbi, and thank you for reporting this to us!

I took a look at the 3 messages in the bufr.zip file you attached, and it turns out the library is correct that there are errors in the encoding of the messages. Specifically, if you run debufr with the -b option, it will print out all of the BUFR descriptors in Section 3 without trying to actually decode the data values in Section 4. When you do this, you can see for yourself that in each of the messages there are occurrences of the descriptor '031001' in the Section 3 descriptor list.

The problem is that '031001' is used solely for storing a delayed replication count, so it's only valid when it immediately follows a delayed replication descriptor of the form '1XX000', and where XX is the number of descriptors to be replicated. So, bottom line, the library is doing the right thing by diagnosing this as a n error, and I'm honestly stumped as to how the on-line decoders you referred to (or, for that matter, any proper BUFR decoder?) could overcome that particular error to make any sense of what's in the message.

Again, the descriptor '031001' is illegal in any context other than immediately following a '1XX000' descriptor, which you can see for yourself isn't present in any of these messages. We've actually already noticed this issue ourselves in some of the data we've received operationally at NCEP/EMC, and we've already separately contacted the folks at the Korea Meteorological Administration ('RKSL' is the originator for Seoul, South Korea) to report this to them and request that they fix it.