opengeos / HyperCoast

A Python Package for Visualizing and Analyzing Hyperspectral Data in Coastal Environments
https://hypercoast.org
MIT License
150 stars 26 forks source link

Error in reading NEON data (other than SERC site example) #50

Closed gponce-ars closed 4 months ago

gponce-ars commented 4 months ago

Environment Information

Description

Error reading NEON files

I tried to read some NEON hyperspectral files (.h5) and got a specific error making reference to a NEON site-related (SERC). The script to read .h5 from NEON hypercoast.read_neon() has this line with specific metadata that shouldn't be there, right (line 64 ?

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[2], line 1
----> 1 dataset = hypercoast.read_neon('NEON_D14_WGEW_DP1_20180905_161438_reflectance.h5')

File /mnt/DATA/SOFTWARE/miniconda3/envs/storm_env/lib/python3.11/site-packages/hypercoast/neon.py:64, in read_neon(filepath, wavelengths, method, **kwargs)
     48 """
     49 Reads NEON AOP hyperspectral hdf5 files and returns an xarray dataset.
     50 
   (...)
     60     xr.Dataset: The dataset containing the reflectance data.
     61 """
     62 f = h5py.File(filepath, "r")
---> 64 serc_refl = f["SERC"]["Reflectance"]
     65 wavelengths = serc_refl["Metadata"]["Spectral_Data"]["Wavelength"][()].tolist()
     66 wavelengths = [round(num, 2) for num in wavelengths]

File h5py/_objects.pyx:54, in h5py._objects.with_phil.wrapper()

File h5py/_objects.pyx:55, in h5py._objects.with_phil.wrapper()

File /mnt/DATA/SOFTWARE/miniconda3/envs/storm_env/lib/python3.11/site-packages/h5py/_hl/group.py:357, in Group.__getitem__(self, name)
    355         raise ValueError("Invalid HDF5 object reference")
    356 elif isinstance(name, (bytes, str)):
--> 357     oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
    358 else:
    359     raise TypeError("Accessing a group is done with bytes or str, "
    360                     "not {}".format(type(name)))

I have a NEON file in my local drive and this is what I typed:

$python
Python 3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:40:35) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hypercoast
>>> dataset = hypercoast.read_neon('NEON_D14_SRER_DP1_20180824_163411_reflectance.h5')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/DATA/SOFTWARE/miniconda3/envs/storm_env/lib/python3.11/site-packages/hypercoast/neon.py", line 64, in read_neon
    serc_refl = f["SERC"]["Reflectance"]
                ~^^^^^^^^
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/mnt/DATA/SOFTWARE/miniconda3/envs/storm_env/lib/python3.11/site-packages/h5py/_hl/group.py", line 357, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: "Unable to synchronously open object (object 'SERC' doesn't exist)"

I guess just need to modify that line and replace that 'SERC' with the acronym of my site.

giswqs commented 4 months ago

Thank you for trying it out. Would you like to submit a pull request to fix the big? Appreciated.

giswqs commented 4 months ago

Or can you share the sample dataset so that I can inspect the metadata that's different from the SERC datasets?

gponce-ars commented 4 months ago

Thank you for trying it out. Would you like to submit a pull request to fix the big? Appreciated.

Sure...