search

cambecc / grib2json

converts GRIB2 files to JSON
MIT License
356 stars 154 forks source link

Values truncated #25

Closed heffrey78 closed 7 years ago

heffrey78 commented 7 years ago

I've been using another grib api and need to switch. Comparing results from the previous api with what grib2json offers, I'm getting less decimal precision. I was getting 14 digits to the right of the decimal. Now I'm getting 7. Is there a parameter with which I can change this or am I going to need to dig deeper?

cambecc commented 7 years ago

This sounds a lot like the difference in precision between single and double precision floating point values. There is no switch to support double precision, currently. However, does your data set really require double precision? Perhaps single precision will be good enough for your needs.

heffrey78 commented 7 years ago

That was the direction I was headed. Luckily the client said that they did not need that level of precision so I'm off the hook. Thanks for the response.

cambecc commented 7 years ago

👍