Wave Hindcast documentation -- Can not run examples

[brunj7]

Hi,

I am trying to run the examples on this page: https://github.com/openEDI/documentation/blob/main/US_Wave.md but I am having trouble with accessing the data (getting the metadata works). Is it still up-to-date?

Thank you for any input!

from rex import ResourceX

wave_file = '/nrel/US_wave/West_Coast/West_Coast_wave_2010.h5'
with ResourceX(wave_file, hsds=True) as f:
    meta = f.meta
    time_index = f.time_index
    swh = f['significant_wave_height']

Error message:

Traceback (most recent call last):
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/h5pyd/_hl/dataset.py", line 1185, in __getitem__
    rsp = self.GET(req, params=params, format="binary")
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/h5pyd/_hl/base.py", line 979, in GET
    raise IOError("no data returned")
OSError: no data returned

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File "<stdin>", line 2, in <module>
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/rex/resource_extraction/resource_extraction.py", line 219, in meta
    return self.resource.meta
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/rex/resource.py", line 786, in meta
    self._meta = self._get_meta('meta', slice(None))
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/rex/resource.py", line 1175, in _get_meta
    meta = ResourceDataset.extract(meta, sites, unscale=False)
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/rex/resource.py", line 532, in extract
    return dset[ds_slice]
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/rex/resource.py", line 52, in __getitem__
    return self._get_ds_slice(ds_slice)
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/rex/resource.py", line 502, in _get_ds_slice
    out = self._extract_ds_slice(ds_slice)
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/rex/resource.py", line 447, in _extract_ds_slice
    out = self.ds[slices]
  File "/Users/brun/GitHub/gitNCEAS/ca-mpa/.venv/lib/python3.9/site-packages/h5pyd/_hl/dataset.py", line 1194, in __getitem__
    raise IOError(f"Error retrieving data: {ioe.errno}")
OSError: Error retrieving data: None

[jgu2]

Apologies for the very late response.

The repo and example is up-to-date, and it work after a test from my side.

Firstly, make sure the libraries and configurations are in good shape.

• rex installation reference - https://github.com/NREL/rex
• To use rex with HSDS you will need to install h5pyd, pip install h5pyd.

• configure hsds service by running hsconfigure, and enter the following prompts

  $ hsconfigure
  Enter new values or accept defaults in brackets with Enter.
  
  Server endpoint [https://developer.nrel.gov/api/hsds]:
  Username [None]:
  Password [None]:
  API Key [3K3JQbjZmWctY0xmIfSYvYgtIcM3CN0cb1Y2w9bf]:
  Testing connection...
  connection ok
  Quit? (Y/N)Y

  The connection looks good.

Next, run the example mentioned

from rex import ResourceX

wave_file = '/nrel/US_wave/West_Coast/West_Coast_wave_2010.h5'

with ResourceX(wave_file, hsds=True) as f:
    meta = f.meta
    time_index = f.time_index
    swh = f["significant_wave_height"]

    print(swh)

The result look like this

[[ 1.85869  2.28684  2.44619 ...  0.      -9.       0.     ]
 [ 1.85921  2.39396  2.40333 ...  0.      -9.       0.     ]
 [ 1.90968  2.26035  2.28689 ...  0.      -9.       0.     ]
 ...
 [ 0.86439  2.00867  2.11963 ...  0.      -9.       0.     ]
 [ 0.88042  1.95738  2.07173 ...  0.      -9.       0.     ]
 [ 0.9651   1.9153   2.03774 ...  0.      -9.       0.     ]]

The dataset loading is relatively slow, a few mins, because it's trying to load all significant_wave_height data from all chunks, this file size of West_Cost_wave_2010.h5 is about 90GB.

Instead, you may test with f.attrs, f.datasets, or f.meta ... which retrieves the metadata information of the dataset, and would be faster, for example,

from rex import ResourceX

wave_file = '/nrel/US_wave/West_Coast/West_Coast_wave_2010.h5'

with ResourceX(wave_file, hsds=True) as f:
    print(f.datasets)

The result would be,

['coordinates', 'directionality_coefficient', 'energy_period', 
'maximum_energy_direction', 'mean_absolute_period', 
'mean_wave_direction', 'mean_zero-crossing_period',
 'meta', 'omni-directional_wave_power', 'peak_period', 
'significant_wave_height', 'spectral_width', 'time_index', 
'water_depth']

After install rex package, there's also US-wave command integrated to interactive with US wave dataset,

$ US-wave --version
US-wave, version 0.2.84

For more information, please refer to the docs here https://nrel.github.io/rex/_cli/US-wave.html#US-wave