Closed ssolson closed 3 years ago
Below is my proposed solution for the ndbc.to_mhkit
format. This is essentially a functionalized version of Step 3 in the example given at the top of #53.
The highlights:
[ndbc_data[year] = to_mhkit(parameter, ndbc_data[year]) for year in ndbc_data]
) so the overhead on the user is still quite small in my opinion. stdmet
(e.g. 46022 year 2019 stdmet
) this questions the original decision because for the 'stdmet' parameter it would be most intuitive to users and easiest to manipulate the data if the data were returned with a DateTime index river
and tidal
modules. Let me know what you guys think and we can move forward with getting this function implemented!
# Set the parameter to be spectral wave density for NDBC buoy 46022
parameter='swden'
buoy_number='46022'
# Find available parameter data for NDBC buoy_number
available_data= ndbc.available_data(parameter,
buoy_number)
# Get dictionary of parameter data by year
filenames= available_data['filename']
ndbc_data = ndbc.request_data(parameter, filenames)
# Iterate over each Dictionary key (e.g. 'year')
for year in ndbc_data:
# Get the DataFrame of the year
year_data = ndbc_data[year]
# Create a Datetime Index and remove NOAA date columns for each year
ndbc_data[year] = to_mhkit(parameter, year_data)
def to_mhkit(parameter, df):
'''
Converts the parameter DataFrame to the format expected by mhkit by
modifying the passed DataFrame by removing the NDBC date columns,
and setting a datetime index
Parameters
------------
parameter: string
'swden' : 'Raw Spectral Wave Current Year Historical Data'
'stdmet': 'Standard Meteorological Current Year Historical Data'
df: DataFrame
NDBC data in dataframe to be converted
Returns
-------
ndbc_data: DataFrame
Dataframe with NDBC date columns removed, and datetime index
'''
year_data['date'], ndbc_date_cols = dates_to_datetime(parameter,
year_data,
return_date_cols=True)
year_data = year_data.drop(ndbc_date_cols, axis=1)
if parameter=`swden`:
# Convert columns to float now that the ndbc_date_cols (type=str) are dropped
year_data.columns = year_data.columns.astype(float)
year_data = year_data.set_index('date')
return ndbc_data
Add functionality to convert requested NDBC data to DateTime format and configure the data for use in MHKiT. The example in #53 shows a for loop which sets a DateTime index and removes the NDBC columns. This is a required step to use the data in MHKiT. Functionality that performed the conversion for the user would be useful as this will be a common step users will want to take.
An open discussion point is if historical data should be indexed by time. Currently, MHKiT standard format expects data to be increased by column. So for time domain and frequency domain spectral data, the index would be the frequency bins and the columns would be time and spectral density respectively. For this spectral data, this allowed MHKiT to specify a standard format. However, as MHKiT starts to incorporate more data (e.g. NDBC historical standard meteorological data) this would mean that the index is now wind speed, wind direction, etc. and the columns are increasing datetimes.
Make Sure: [ ] For parameter=
swden
make sure to convert spectral frequency bins from str to float after dropping NDBC date columns