search

Reading-eScience-Centre / edal-java

Environmental Data Abstraction Layer libraries
Other
39 stars 30 forks source link

Wrong rendering of data having NaN values #109

Open mtsales opened 6 years ago

mtsales commented 6 years ago

I have datasets containing NaN values. These NaN values are rendered as a small value (1?) instead of being ignored

guygriffiths commented 6 years ago

OK, can you provide some data that you're seeing this issue with?

mtsales commented 6 years ago

Here it is

cru_TempMonthly_tmp_2015.zip

guygriffiths commented 6 years ago

This looks like a bug in the NetCDF libraries we are using. I will report it to them and integrate their new versions once they fix it.

For further information, the data you supplied does not contain any NaN values. It contains integer values which use 99999 as a fill value, and which are scaled by a factor of 1e-5. The problem occurs when the CDM method tries to check whether the value is missing. The value being tested has the scaling applied (so 99999 becomes 0.99999), but is then tested against 99999.

mtsales commented 6 years ago

Thanks! Much appreciated. I was checking the data in Panoply and there it shows as NaN. Sorry for the misinformation.

guygriffiths commented 6 years ago

No problem, missing data tends to appear as NaNs once it has been processed, so it's a natural assumption to make. The extra information was primarily for benefit of the Unidata developers, since I've given them a link to this bug report so they have access to some test data.

lesserwhirls commented 6 years ago

Fixed in 5.0.0-beta2 :-) Thanks for the report!

guygriffiths commented 6 years ago

Thanks @lesserwhirls, that's been updated in develop and is all working nicely with the test dataset.