MHKiT-Software / MHKiT-Python

MHKiT-Python provides the marine renewable energy (MRE) community tools for data processing, visualization, quality control, resource assessment, and device performance.
https://mhkit-software.github.io/MHKiT/
BSD 3-Clause "New" or "Revised" License
47 stars 45 forks source link

Cannot access WPTO Hindcast Dataset since Friday 6/13/2023 #246

Closed radityadanu closed 1 year ago

radityadanu commented 1 year ago

Describe the bug:

I had the script to access the WPTO hindcast dataset using the wave module as described in example and it worked before. However, the script is now spit error which I could not understand. I also tried to access the dataset using Marine Energy Atlas website (https://maps.nrel.gov/marine-energy-atlas/) and came up with no luck.

To Reproduce:

The same example from examples/WPTO_hindcast_example.ipynb.

` data_type = '3-hour' # setting the data type to the 3-hour dataset years = [1995] lat_lon = (44.624076,-124.280097) parameter = 'significant_wave_height'

Hs, metadata= wave.io.hindcast.hindcast.request_wpto_point_data(data_type,parameter,lat_lon,years) `

image

Expected behavior:

It is expected to return a timeseries dataset for specific spatial boundaries.

Screenshots:

image

Desktop (please complete the following information):

ssolson commented 1 year ago

Hey @radityadanu thank you for your interest in MHKiT.

This issue is most likely related to the API not returning data. This is not MHKiT but the server that hosts the data. I am working to make the API return as reliant as possible by cacheing data and making smaller requests.

Just today you can find my latest comment discussing this issue for the wind data in #241. For context the Wind hindcast data is stored on the same server you are trying to access.

I will look into your specific issue later this week and see if I have any luck.

@rpauly18 & @akeeste just wanted to put this on your radar that others continue to have issues accessing the data (not only from MHKiT but also from Atlas) pointing to a continuation of issues with the data hosting.

As I said I will follow up but I also wanted to thank you again because indicating your problem here is helpful for us to get resources and attention to address the data hosting issues.

ssolson commented 1 year ago

Hey @radityadanu I took a look at the example issue and I can confirm the issue is the API as I expected. So to replicate this exact call I can use NREL-rex (which wraps h5pyd) as follows:

from rex import WaveX
data_path = '/nrel/US_wave/West_Coast/West_Coast_wave_1995.h5'
with WaveX(data_path, hsds=True) as rex_wave:
    ds = rex_wave['significant_wave_height', 1, 1]

This should return a single value of Hs, but the API will return nothing.

Then to show the problem is not REX I can use just h5pyd to do the same thing

import h5pyd
data_path = '/nrel/US_wave/West_Coast/West_Coast_wave_1995.h5'
f = h5pyd.File(data_path, 'r')
dset = f['significant_wave_height']
dset[1, 1]

This again returns no data.

So in conclusion if the API worked MHKiT would return the data as expected. I am actively working on solving this issue with the dataset host but the solution to your problem is outside of the scope of MHKiT.

ssolson commented 1 year ago

Raised the issue with the NREL development folks

https://github.com/NREL/developer.nrel.gov/issues/320

radityadanu commented 1 year ago

Hi @ssolson, thank you for your fantastic work! I tried your solution, and I can confirm that the problem is in the API. There are times that I can access the data, but the error occurs randomly again.

Thank you also for raising the issue to the correct desk! I think I could close this issue and follow up with https://github.com/NREL/developer.nrel.gov/issues/320