NOAA-EMC / wgrib2

Provides functionality for interacting with, reading, writing, and manipulating GRIB2 files.
41 stars 13 forks source link

import_ieee producing corrupted gribs #288

Open Plantain opened 3 weeks ago

Plantain commented 3 weeks ago

import_ieee seems to have started producing corrupted/invalid/nonsense gribs since 2.0.8.

Input data: repro.tgz Reproduction command: wgrib2 -append template.grb2 -little_endian -import_ieee myvar.bin -set_metadata myvar.inv -grib_out out.grb2

Any version after 2.0.8 seems to produce an invalid/wrong GRIB that is blank with occasional bands of values way higher than the data Screenshot 2024-11-04 at 2 12 51 pm Should look like as below, as produced on 2.0.8: Screenshot 2024-11-04 at 2 12 29 pm

diff'ing the grib_dump is interesting:

67,68c67,68
<   scaleFactorOfFirstFixedSurface = -1;
<   scaledValueOfFirstFixedSurface = 255;
---
>   scaleFactorOfFirstFixedSurface = MISSING;
>   scaledValueOfFirstFixedSurface = MISSING;
75c75
<   level = 2550;
---
>   level = 0;
86,90d85
<   14.6875, 12.375, 10.4688, 8.9375, 7.75,
<   6.96875, 6.75, 7.03125, 7.625, 8.3125,
<   9, 9.46875, 9.5625, 9.65625, 9.71875,
<   9.71875, 9.34375, 8.46875, 6.96875, 4.65625,
<   1.21875, 0, 0, 0, 0,
97,100d91
<   0, 0, 0, 1.6875, 5.0625,
<   8.3125, 11.4375, 14.5938, 17.8438, 21.0625,
<   24.2812, 27.3125, 30.25, 32.9375, 35.6562,
<   38.3438, 41.1562, 43.8125, 46.375, 0,
103,105c94,105
<   0.59375, 2.8125, 5.25, 7.875, 10.625,
<   13.5625, 16.6562, 19.75, 22.7812, 25.8125,
<   28.8125, 31.875, 35.2188, 39.0312, 43.0625
---
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0,
>   0, 0, 0, 0, 0
108c108
<   #-READ ONLY- maximum = 100;
---
>   #-READ ONLY- maximum = 101856;
110,113c110,113
<   #-READ ONLY- average = 3.14904;
<   #-READ ONLY- standardDeviation = 11.7847;
<   #-READ ONLY- skewness = 4.65644;
<   #-READ ONLY- kurtosis = 23.4872;
---
>   #-READ ONLY- average = 97296.9;
>   #-READ ONLY- standardDeviation = 19624.7;
>   #-READ ONLY- skewness = -4.75568;
>   #-READ ONLY- kurtosis = 20.6191;
webisu commented 3 weeks ago

I was able to duplicate and fix the problem. If you are using x86_64 (i.e. intel or amd cpu), you can use "-import_bin file.bin" instead of "-little_endian -import_ieee file.bin". The import_bin is faster than import_ieee. Also works with any little-endian computer with 4-byte integers and ieee floats.