search

MazamaScience / MazamaSatelliteUtils

Satellite Data Download and Utility Functions
http://mazamascience.github.io/MazamaSatelliteUtils
GNU Lesser General Public License v3.0
0 stars 0 forks source link

AOD values not being read correctly from NetCDF #72

Closed lagerratrobe closed 4 years ago

lagerratrobe commented 4 years ago

We are seeing AOD data values which fall well outside the stated range of [-0.05, 5] Specifically, we see many values lower than -2. This should not be possible. Also, there seem to be no values greater than 2.4 which seems unlikely given the geographic regions with heavy smoke that we are looking at.

The cause for this is that we are not properly reading AOD values from the NetCDF file. Negarive values in the raw data are not being properly read as unsigned shorts, thus causing a shifted range of values to be produced. This is due to a bug in the ncdf4 R library we are using.

We need to change the ncvar_get call for AOD and do the following:

  1. Read the data "raw" using the rawdatavals = FALSE_ option
  2. Set all -1 values to NA
  3. Add 65536 to all remaining negative numbers
  4. Apply the Scaling and Offset values
lagerratrobe commented 4 years ago

Since I have some test files created to prove that there was a problem, I can also use them to test the fix for this. And I really want a test case, because this has bitten us once already.