search

Unidata / netcdf-java

The Unidata netcdf-java library
https://docs.unidata.ucar.edu/netcdf-java/current/userguide/index.html
BSD 3-Clause "New" or "Revised" License
143 stars 68 forks source link

Grib2 reader issue for data using second order packing and a bitmap section #1305

Open team-metoc opened 5 months ago

team-metoc commented 5 months ago

Versions impacted by the bug

All versions up to and including netcdf-java-5.5.3

What went wrong?

Our team is currently preparing to read Grib2 files featuring oceanographic data encoded with ECMWF's second order packing. However it seems that, while the second order packing type is fully supported by netCDF-Java, the library's read process does not take into account the presence of a bitmap section. Reading the source code, we think this is because Grib2DataReader.getData50002 method doesn't use the bitmap array (*) (while other packing types like 40 - JPEG2000 do and work perfectly on oceanographic data). For this reason, we think any attempt to read a Grib2 file featuring oceanographic data encoded using second order packing and a bitmap section will either fail or produce a random array of points.

(*) To be more precise, at the end of the method, the loop converting the data to the return array should be using the GribNumbers.testIsBitSet method (like in Grib2DataReader.getData40 for example)

Thank you for your help.

Relevant stack trace

No response

Relevant log messages

No response

If you have an example file that you can share, please attach it to this issue.

If so, may we include it in our test datasets to help ensure the bug does not return once fixed? Note: the test datasets are publicly accessible without restriction.

No

Code of Conduct

haileyajohnson commented 5 months ago

Thanks for opening an issue! Unfortunately, the netcdf-java maintainers are not necessarily experts in all the data formats that we support, as much as we would like to be. I'm not sure that we will be able to look at this issue unless you're able to provide a test file and an example of a failure?

I have also added the help-wanted tag to indicate we'd love the larger community to take a look.

team-metoc commented 4 months ago

Thank you for your answer. We are providing a sample grib and a screenshot of the problem from toolsUI : grib2-read-error sea_surface_temperature.zip

The data is sea surface temperature, so we should be seeing colored points only at sea.

Thank you for your help.