[5.5.3]: UnsupportedOperationException while reading a variable from BUFR, but no error in NCdumpW

[JP95Git]

Versions impacted by the bug

v5.x

What went wrong?

After updating the BUFR tables to the latest version (see #1218), I was able to read almost all BUFR files. The attached file can't be read. I get the exception as seen in "Relevant stack trace".

Attached BUFR file: Z__C_SOWR_20230723011100_aero1,IUSD02_SOWR_230000____RSSO.zip

Reading is done with this code:

Logger log = LoggerFactory.getLogger("logger");

String filename = "Z__C_SOWR_20230723011100_aero1,IUSD02_SOWR_230000____RSSO.bin";

try (NetcdfDataset ncFile = NetcdfDatasets.openDataset(filename);) {
    VariableDS rawValue = (VariableDS) ncFile.findVariable("obs.WMO_block_number");
    Integer value = rawValue.readScalarInt(); // <- exception happens here
    System.out.println("Value: " + value);
} catch (Exception e) {
    log.error("error:", e);
}

Using your NCdumpW I can see all variables without the error using this code:

String filename = "Z__C_SOWR_20230723011100_aero1,IUSD02_SOWR_230000____RSSO.bin";

try (Writer writer = new BufferedWriter(new FileWriter(filename + ".txt"));
        NetcdfFile nc = NetcdfDatasets.openFile(filename, null)) {
    NCdumpW.print(nc, "-vall ", writer, null);
}

Relevant stack trace

java.lang.UnsupportedOperationException: not a scalar variable =byte WMO_block_number;
  :missing_value = 127B; // byte
  :BUFR:TableB_descriptor = "0-1-1";
  :BUFR:bitWidth = 7; // int

    at ucar.nc2.Variable._readScalarData(Variable.java:777)
    at ucar.nc2.Variable.readScalarInt(Variable.java:651)

Relevant log messages

No response

If you have an example file that you can share, please attach it to this issue.

If so, may we include it in our test datasets to help ensure the bug does not return once fixed? Note: the test datasets are publicly accessible without restriction.

Yes

Code of Conduct

• [X] I agree to follow the UCAR/Unidata Code of Conduct

[mnlerman]

Hello, I was able to run the code that you provided and I can see that rawValue.readScalarInt() throwing an UnsupportedOperationException is the expected behavior in this case. According to the docs:

readScalarInt throws UnsupportedOperationException - if not a scalar Variable or one-dimensional of length 1.

rawValue contains a one-dimensional array of length 2 [12, 12]. You can get an array with the data in rawValue by calling rawValue.read()

[JP95Git]

Thanks for your help.

According to this tool https://kunden.dwd.de/bufrviewer/ the BUFR file has 2 submessage, Wikipedia called them subsets.

I managed to read both submessages or subsets for some variables:

// For readScalarInt

Array arrayData = rawValue.read().reduce();
for (int i = 0; i <= arrayData.getRank(); i++) {
    log.debug("Value[{}]: {}", i, arrayData.getInt(i));
}

// For readScalarString

Array arrayData = rawValue.read().reduce();
DataType dataType = rawValue.getDataType();
if (dataType == DataType.STRING) {
    for (int i = 0; i < arrayData.getRank(); i++) {
        log.debug("Value[{}]: {}", i, arrayData.getObject(i));
    }
} else if (dataType == DataType.CHAR) {
    ArrayChar arrayDataChar = (ArrayChar) arrayData;
    for (int i = 0; i < arrayDataChar.getRank(); i++) {
        log.debug("Value[{}]: {}", i, arrayDataChar.getString(i));
    }
}

It's worth to note that some variables have more values than the others. E.g. I my BUFR files has 2 WMO_station_number but only 1 Solar_and_infrared_radiation_correction. So importing such a file will be not so easy.

It would be cool if I could open one BUFR file, iterate over the submessages and then read all variables of these submessage.