Value out of range with MODES

[dcesari]

Again an issue with this kind of aircraft report MODE_12.zip:

wrep MODE_12.bufr
MODE_12.bufr:0:Value 15 is outside the range [0,14] for 008009 (DETAILED PHASE OF FLIGHT)

Actually B08009 has 4 bit width and zero offset and scale, I guess from this comment in varinfo.cc

                int bit_max = exp2(bit_len) + bit_ref;
                // We subtract 2 because 2^bit_len-1 is the
                // BUFR missing value.
                // We cannot subtract 2 from the delayed replication
                // factors because RADAR BUFR messages have 255
                // subsets, and the delayed replication field is 8
                // bits, so 255 is the missing value, and if we
                // disallow it here we cannot import radars anymore.
                if (WR_VAR_X(code) != 31)
                    bit_max -= 2;

that this is a desired behavior and 15 should be a missing value. OTOH bufr_dump -p allegedly reports

detailedPhaseOfFlight={
      3, 3, 15, 3, 3, 3, 3, 3, 3, 3, 
      6, 3, 3, 3 }

so, first we should understand whether that is meant to be a missing value or not (but that's not @spanezz job) and, second, assuming it is really intended as a missing value, is there a way (environmental variable?) to avoid raising the error and code it as missing in (bufr2)netcdf?

[dcesari]

I found this long dballe issue, https://github.com/ARPA-SIMC/dballe/issues/241 partly related, so adding to bufr2netcdf the --domain-errors option as in dballe and setting it to unset could probably help in this case.

However this looks really like a case where the value was deliberately coded as missing/unset (phase of flight), so, generally speaking, I feel that val == imax + 1 (missing value?) should be treated differently from val < imin || val > imax + 1 (domain error), although, since bufr2netcdf just does plain bufr decoding without interpretation or conversion, val = imax + 1 is probably the only possible source of domain error?!

[spanezz]

When all bits in one value in BUFR are set to 1, it automatically means missing value, and wreport should be able to detect that, and does it all the time. I'll now try to figure out why it is not doing it right this time

[spanezz]

Found. It is a compressed BUFR, and wreport treats as missing:

• a base value set to all 1s
• a difference set to all 1s In this case, it's base + difference that is set to all 1s.

It looks to me like an issue with the encoder that was used, which explains why bufr_dump also computes 15 (value out of range) and wrep raises a domain error.

I am going to change it so that if base+difference is set to all 1s, then wreport also treats that case as a missing value, because a computed value of all 1s, given how BUFR works, can never be a valid value anyway, and we are making the decoder more tolerant