Hindcast Module – Loading Data

[mankleh]

I have been running into issues with the Hindcast module when attempting to run using multiple years (generally over 5/6 years). I often run into the 429 Error “Too many Requests.”

Because of this error I was hoping to download the .h5 files from AWS and load files locally by turning the hsds flag to False. When I try to do this and run the function I get the error “FileInputError: Could not find any file paths with pattern: /nrel/US_wave/virtual_buoy/West_Coast/West_Coast_Virtualbuoy*.h5”

I tried running this function with the files in the same folder as the jupyter-notebook file that is calling the data as well as creating a folder with the same path name as the error above. In both instances the hindcast.request_wpto_point_data() function doesn’t recognize the .h5 files trying to be called.

[ssolson]

Thank you for your interest in MHKiT mankleh. Could you please provide a minimum working example that replicates this issue for us to look at?

[mankleh]

hindcastExamples.zip

I have a zip drive with two Jupyters files for each mentioned issue. I couldn't include the downloaded .h5 file due to size, but the example with the file loading issue I tried to locate "West_Coast_virtual_buoy_2005.h5" downloaded from AWS.

[camirmas]

@ssolson The issue with the local files is that Python's glob function wants a relative or absolute path, which causes an error for a path like /nrel/US_wave/virtual_buoy/West_Coast/West_Coast_Virtual_buoy_*.h5 when hsds=False. You can see its use in rex, where the error occurs.

I've created a fork of this repo with a quick change that allows users to specify a custom .h5 path. I've only implemented it for the waves module for now, but I'd be happy to apply it elsewhere and submit a PR if the team is interested in adding it as a feature. Let me know what you think.

https://github.com/camirmas/MHKiT-Python/commit/10d0ce5393ef39c4e1c925343a8e8ca892719ce2

[ssolson]

@rpauly18 what are your current thoughts on this issue?

[ssolson]

Thank you @camirmas I added your suggestions to #220 which should be moved into the development branch soon. I apologize for the extreme delay.