Installing WDRT package into python

[andreowhite]

Hello:

I am now in the process of learning to use WDRT. In particular, I would like to use the ESSC module. However, I don't know how to get this module installed into python. I tried to indtall using conda but it did not find the WDRT package.

Please let me know what to do.

Thank you. Regards, AOAW

[ssolson]

Hey @andreowhite and thank you for your interest in WDRT. Currently WDRT is not a package registered with PyPI and therefore you must download it and do a manual install (we are currently in the process of migrating the WDRT code into MHKiT a package on PyPI ). In the meantime to install this package you will want to navigate to the WDRT GitHub and click the green code button on then click download zip.

After extracting this zip using the "Anaconda Prompt" (search for this in windows) we can navigate to the location where you unzipped the folder using the "cd " (change directories) command cd path/to/unzipped/folder for me this was 'C:\Users\ssolson\Downloads\WDRT-master\WDRT-master'

Then we can install WDRT using pip. In the Anaconda Prompt you will type: pip install . where the "." means here.

You should see something like this:

I apologize that our installation instructions (https://wec-sim.github.io/WDRT/installation.html) are currently outdated. I will update the documentation there before I close this issue.

Let me know if you have any more issues or if these instructions solve your problem

[andreowhite]

Hello @ssolson thanks for the feedback. I just have one problem. I do not know how to extract file using anaconda prompt. I have googled but not seeing a clear command to do that.

[ssolson]

Sorry for the confusion. The extraction of the zip should be done using your system zip extractor. I am not sure what OS you are using but simply extract the zip as you normally would. Then cd to the extracted zip location in the Anaconda Prompt.

[andreowhite]

Hi @ssolson I have got this error:

Processing c:\users\awhite15\documents\wdrtmaster ERROR: Command errored out with exit status 1: command: 'C:\Users\awhite15\AppData\Local\Continuum\anaconda2\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'c:\users\awhite15\appdata\local\temp\pip-req-build-adlxuq\setup.py'"'"'; file='"'"'c:\users\awhite15\appdata\local\temp\pip-req-build-adlxuq\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info cwd: c:\users\awhite15\appdata\local\temp\pip-req-build-adlxuq\ Complete output (5 lines): Traceback (most recent call last): File "", line 1, in File "c:\users\awhite15\appdata\local\temp\pip-req-build-adlxuq\setup.py", line 34, in with open(os.path.join(file_dir, 'README.md'), encoding='utf-8') as f: TypeError: 'encoding' is an invalid keyword argument for this function

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

[andreowhite]

Hi @ssolson I have successfully installed from the windows command prompt.

[andreowhite]

Many thanks.

[ssolson]

Okay, so you needed to use the command prompt instead of the Anaconda prompt for your issue? I default to the Anaconda Prompt for Anaconda users because by default the Anaconda installation wizard does not add Python to your system PATH (e.g. Windows command prompt may not know where python.exe is located).

[andreowhite]

Hi @ssolson the WDRT package still has not been installed. This the message I got in the windows command prompt, Installing collected packages: WDRT Attempting uninstall: WDRT Found existing installation: WDRT 0.1.0 Uninstalling WDRT-0.1.0: Successfully uninstalled WDRT-0.1.0 Successfully installed WDRT-0.1.0

Which I believe is an indication that it has been installed. I used 'pip install .' as you suggested. However, when I try to import WDRT.ESSC as ESSC I get an error indicating that the module is not installed.

I tried again to install as the per the instructions of the site using "python 2 setup.py and het this error: Traceback (most recent call last): File "setup.py", line 34, in with open(os.path.join(file_dir, 'README.md'), encoding='utf-8') as f: TypeError: 'encoding' is an invalid keyword argument for this function.

The solutions that they have said worked is this:

a) For your issue -- in Python 2, the open() function takes no encoding argument (the third argument is the buffering option instead).

The fix is to add the following line near the top of the vaderSentiment.py file: from io import open

b) open('textfile.txt', encoding='utf-16') Use io, it will work in both 2.7 and 3.6 python version

import io io.open('textfile.txt', encoding='utf-16') I can see that it is giving an instruction to the setup.py file to make a change to the text I think but I don't know how to make the modification that they are proposing.

I am not sure if you are familiar with this error and if you can help to solve it. If you don't have any solution please let me know. If there is no soultion to this problem, I will move to another software to carry out the analysis.

Thanks.

Regards, AOAW

[ssolson]

Hey @andreowhite can you confirm if the example files (in the example folder) run? If one does not run what is the error?

This package is now fully Python 3 and no longer supports Python 2 (again apologies on the outdated documentation), please make sure you are using Python 3 to run WDRT.

If the examples files are giving you an error we could schedule a call to discuss.

[andreowhite]

Hey @ssolson thanks for that advice. It worked for python3. I just spent the last couple days downloading python2 and going through a lot of rigmarole because the instructions on the site specifically states that python2.7.x is to be used and not 3.x.x.

I just happen to try again to run the example again a second before your reply and it worked.

Many thanks.

I will be in touch again if I have any further queries since I will now be using my site data.

[andreowhite]

@ssolson I had used python 3 in error to run the example and boom it worked. Then your message came in.

[andreowhite]

Hi @ssolson I am now going through the ESSC modue and saw the following statement: "The Extreme Sea State Contour (ESSC) module contains the tools necessary to calculate environmental contours of extreme sea states for buoy data..

Does WDRT only work for buoy data? In which case, it does not work for reanalysis data.

[ryancoe]

@andreowhite - Thanks for your interest. The ESSC module is mainly designed to work with the buoy data (i.e., all of the import functionality works with the CDIP and NDBC buoys). Nonetheless, there's nothing stopping you from working with other data (e.g., hindmost/reanalysis data). You will simply need to take a look at how the ESSC module expects the data to be input. You would effectively be creating a Buoy object that is populated via your modeled data.

https://github.com/WEC-Sim/WDRT/blob/44867b83a5b13e35f9a223e6e373fbb71a138806/WDRT/ESSC.py#L2707

[andreowhite]

Hi @ryancoe I have found a script from @aubreyeckert that was done in 2017 that addresses this. I have shown it below because I have one query about the data that is read in as that was not shown in the code. The data that I have a Hs and Tp not energy period, Te. The data has two variables in a text file, Hs and Tp. The Hs value below states that the variable name is HS_Plocan16. I will assume that that is the name of the variable in the .txt file that was read in.

Please advise if a simple text file can be used to read in the data and the buoy object created make equal to it. Such that if I say for example buoy5 = ESSC.buoy('5').

Then buoy5.Hs= variable name for Hs in the text file

# Import packages
import matplotlib.pyplot as plt
import WDRT.ESSC as ESSC

# Read in data from text files as you want

# Create buoy object
buoy16 = ESSC.Buoy('16')

# Set Hs and T directly from data (each must be 1D array)
buoy16.Hs = Hs_Plocan16
buoy16.T = T_Plocan16

# Create PCA EA object
pca16 = ESSC.PCA(buoy16)

# Declare required parameters
Time_SS = 1.  # Sea state duration (hrs)
Time_R = 100  # Return periods (yrs) of interest

# Calculate contour
pca_Hs_Return, pca_T_Return = pca16.getContours(Time_SS, Time_R)

plt.figure()
plt.plot(buoy16.T, buoy16.Hs, 'bo', alpha=0.1, label='PLOCAN16 data')
plt.plot(pca_T_Return, pca_Hs_Return, 'k-', label='100 year contour')
plt.legend(loc='lower right', fontsize='small')
plt.grid(True)
plt.xlabel('Energy period, $T_e$ [s]')
plt.ylabel('Sig. wave height, $H_s$ [m]')
plt.savefig('PLOCAN16_Contour.png',dpi=300)
plt.show()

If you are able to get the Hs and T values for any buoy, you can follow this workflow to calculate the contours. Let me know if this works for you or if you run into any additional issues.

[andreowhite]

@ryancoe Another matter. Regarding the Tp, is it okay to use as I noted that WDRT was set up to use Te and not Tp. But it is the Tp values that I would like to read in. If you could also guide me reading the fitting of distribution to the data (Hs and Tp)is there a function in WDRT to do this, so that I can see that the distribution fits the data?

AOAW

[ryancoe]

@ryancoe Another matter. Regarding the Tp, is it okay to use as I noted that WDRT was set up to use Te and not Tp. But it is the Tp values that I would like to read in. If you could also guide me reading the fitting of distribution to the data (Hs and Tp)is there a function in WDRT to do this, so that I can see that the distribution fits the data?

This is a statistical model, so using Te vs. Tp (or any other variable) is OK.

[ryancoe]

On your previous question about loading from .txt files, a quick search of the source code reveals that there is a built in method for this: https://github.com/WEC-Sim/WDRT/blob/44867b83a5b13e35f9a223e6e373fbb71a138806/WDRT/ESSC.py#L2886

Alternatively, you can load the data in however you like and assign it as done in the example from @aubreyeckert above.

[andreowhite]

@ryancoe thanks for your response on the .txt file. However I need to clarify something. The script shows: import WDRT.ESSC as ESSC buoy46022 = ESSC.Buoy('46022','NDBC') buoy46022.loadFromText() The name of my .txt file is sitedata.txt . Am I to format the code like this(assuming I maintain the buoy5 nomeenclature): bouy5=ESSC.Buoy('5', 'filename'); not sure about what happens here please clarify. bouy5.loadFromText()

Regards, AOAW

[aubreyeckert]

@andreowhite Thank you for your interest in the WDRT/ESSC module! It is great to see that there are use cases beyond the space in which we have been applying the code.

The data handling functions included in the WDRT are specifically designed to work with either WDRT or CDIP buoys, as noted throughout the documentation of these functions. Because your use case is outside of this scope, you will need to add some functionality to set up your data correctly. I would suggest looking through how the NDBC data is handled in the code and following this as a guide for your development. Ultimately, the contour calculations require the "buoy" object to include vectors of Hs and T, you can either modify the contour methods to fit your parameters or figure out a way to save vectors of your 2-D data to the object. If you transfer the code used to do this for the buoy data that it was intended for over to your paradigm, this should give you a good start.

[andreowhite]

@aubreyeckert thanks for your quick response. But I am very disappointed with your response, as I does not help to solve my problem. It is not the same level of input as you provided to @Wadbled in 2017. I show a part of the response that you provided to him in a similar enquiry below. I am able to follow that code except, I do not know how to read in the data. I am attaching the .txt file to which the script refers. PLOCAN_P16_waves.txt. In looking at the file I am just not sure if this data was cleaned up to create Hs_Polcan16 and T_Polcan16 variables and then read in. This is what I need to know specifically. While I concur that this process will aid in my development, you telling me to figure out a way to save the 2-Data as a vector will not really solve my problem. I have been trying to install this WDRT program since last week Wednesday. The instructions on the website clearly outlines that python2.7.x is to be used. After going through a long rigmarole, I found out today(Jul 7) from @ssolson that WDRT is now run on python3. I am kindly asking you to reconsider your response and see if you can assist me in a better way. My objective is to know how to modify the current data format to which files are read in by the program so that it is suitable for a .txt file. What you have told me above will no doubt cause me to waste more time. Then I will have to come back to you. I am therefore, kindly asking you to reconsider your response and see if you can provide me with the assistance that I really need. Thank you.

# Import packages
import matplotlib.pyplot as plt
import WDRT.ESSC as ESSC

# Read in data from text files as you want

# Create buoy object
buoy16 = ESSC.Buoy('16')

# Set Hs and T directly from data (each must be 1D array)
buoy16.Hs = Hs_Plocan16
buoy16.T = T_Plocan16

# Create PCA EA object
pca16 = ESSC.PCA(buoy16)

# Declare required parameters
Time_SS = 1.  # Sea state duration (hrs)
Time_R = 100  # Return periods (yrs) of interest

# Calculate contour
pca_Hs_Return, pca_T_Return = pca16.getContours(Time_SS, Time_R)

plt.figure()
plt.plot(buoy16.T, buoy16.Hs, 'bo', alpha=0.1, label='PLOCAN16 data')
plt.plot(pca_T_Return, pca_Hs_Return, 'k-', label='100 year contour')
plt.legend(loc='lower right', fontsize='small')
plt.grid(True)
plt.xlabel('Energy period, $T_e$ [s]')
plt.ylabel('Sig. wave height, $H_s$ [m]')
plt.savefig('PLOCAN16_Contour.png',dpi=300)
plt.show()

Related to https://github.com/WEC-Sim/WDRT/issues/5

[andreowhite]

Ok thanks @kmruehl it is related to #5.

[aubreyeckert]

@andreowhite WDRT is an open-source research tool with potential issues/bugs and undeveloped features. Incorporation of new features requires the user to independently develop and work with the core team to use the tool beyond the provided examples. We have taken the action to update the installation documentation, as noted above, to avoid similar confusion in the future, and we apologize for the inconvenience that this may have caused our user community.

Our team has noticed that you have not had too much past activity on Github, but we would suggest that you be more thoughtful in your communications with folks who are providing you with help. Some of your language in this thread borders on abusive.

I would like to take this opportunity to “reset" and attempt to move forward in a productive way. If you can make an attempt at loading in the .txt file on your own, we will gladly help you work through any errors. Here are the needed steps: per the example you cited, read in the data using any text processing function in Python that you prefer (as noted in the previous comment, you can follow the ESSC functions for reading text data with Python as a guide) and save one vector to buoy.Hs and one vector to buoy.T. The remainder of the example shows how this buoy object can then be used to calculate an environmental contour. As to the data in the example, it was provided by the user in question – we did not pre-process the data for this user.

[andreowhite]

@aubreyeckert and team please accept my most sincere apologies. I am very grateful for your help. The communication prior was sent purely out of frustration coupled with the fact that it was late at night and I am trying to make tight deadline. As a matter of course, I usually don't respond to emails when I am tired or frustrated in order to guard against off the cuff responses like what occured yesterday. Having thought about it this morning I decided that I need to "eat humble pie", slow down. and go through with the recommendations that both you and @ryancoe sent. Therefore, I have been working with the NBDC46022.h5 file to see what the data looks like and how to work with these types of files.

To tell you a bit about my background, I am a civil engineer, from Jamaica. Have worked in consultancies throughtout my career. I am currently doing a PhD at University of plymouth looking at floating offshore wind. You are therefore correct. I have little or no experience on Github or the coding arena. In fact, my python experience is a little under 3 weeks.

Please accept my aoplogies again and let me emphasize again that I am very greteful for your assistance. If I have any queries while working through the example I will get back to you.

AOAW

[andreowhite]

Hi @aubreyeckert and @ryancoe I have looked at the example for plotting the contours. I managed to do a contour plot loading in the data as text from the NDBC46022 file in examples/data. However, when I tried with my data by modifying on the NDBC text tiles I got an error if the nmber of columns is less than 6. That is because I instert one column for the Hs data. Hence the columns were YY, MM, DD,hh, Hs. If I had inserted another column for Tp it would have worked. However, I would be very grateful if you could explain the data format of the NDBC text file for me. Making reference to the SWD-46022-1996.txt file. I realize that after the hh there are some time steps (assuming) of 0.01 increments. I would like to know if this is like a gridded data where each time step gives different values for the significant wave height. I would also like to know where the Te values are in the said file.

I tried to load my data as a textfile and convert to numpy array and then slice Hs and Tp but I got an error.
Please see attached pdf file. Pages 2 and 3 show the data loaded and the error(page3). I have also attached the contour plot from the example when I loaded the NDBC data as a text file. I was not able to save the image in the pdf so attached separately.

My preffered alternative me would be to know the format of the NDBC text file, for example SWD-46022-1996, and load my data using the same format that the data is presented in this file

See if you could help me with one of the 2 alternatives. The files are here: trial1WDRT_code.pdf trial_contour_plot.pdf

Many thanks for your help.

Regards, AOAW

[aubreyeckert]

Hi @andreowhite - First, looking at the error that you received, I think that you are almost there. The change that I would try first would be to change lblock 44 to:

buoy16.Hs=np.array(swh) buoy16.Tp=np.array(pwp)

It looks to me like the Hs and Tp values are saved as lists, which is why they don't have a 'mean' attribute (numpy arrays do have this attribute, so the fix suggested should resolve the issue) - leading to the "AttributeError: 'list' object has no attribute 'mean'" message.

As to your question about the NDBC text files - the WDRT code loads in data in terms of spectral wave density and frequency and then calculates Hs and Te for each time step in the file (see for instance the 'saveAsTxt' function or the 'loadFromTxt' function. The data is a recorded time history of wave data at a specific location. This is likely why you are experiencing issues following the exact format of the NDBC data to read in your data which is already saved as Hs and T. Since you don't need to make these extra calculations, you can 'skip over' them by reading in your data as you are and then saving the Hs and T vectors directly to the buoy object.

If you run into issues, you may also want to look at the other attributes of the buoy object (see line 2707) - in particular the way that the time histories are saved. However, I think that with the fix suggested above, the contour calculation you are interested in should likely run successfully.

[andreowhite]

@aubreyeckert many thanks. I will check out your suggestions.

AOAW

[ssolson]

References to Python 2.7 have been removed from the documentation and is live at: https://wec-sim.github.io/WDRT/index.html

I refrained from a complete overhaul as this code is in the process of being migrated.

[andreowhite]

Hi @aubreyeckert I did what you suggested. That is to define Hs and Tp but that did not work. So I created a .h5 file and tried to load it in but still getting an error. As it does not recognize the file, it seems. Inside NDBC46022.h5 is a group called buoy_Data and datasets in the group are Hs, T, dateNum and a subgroup called dateList. It seems like dateList is not one of the required variables. Only Hs, T and dataNum. This I deciphered from the error. Page 4 in this document trial2WDRT_code.pdf shows the error and further queries that I have.

I think the data has to be read in as a .h5 file otherwise, I would be very grateful, if you could let me know if there is another way. If it could be loaded in as a numpy array that would have solved the problem. But I am not sure that can be done.

Let me know if you have any other thoughts about how I could solve it.

Regards, AOAW

[andreowhite]

@aubreyeckert in the first line that should read Hs and Tp as numpy arrays.

AOAW

[aubreyeckert]

Hi @andreowhite. I checked out your script and compared it to the way that the WDRT saves h5 files in lines 3158 to 3168 of the ESSC module. If you would like to use the 'loadFromH5' function, I would extract the code from these lines and use it with your data in place of the variables passed by the function. This includes the creation of the buoy_Data object that you correctly identified as the issue you are encountering. Looking at the error message you received, the error is coming after the file is read (so it does recognize it) and is expressed as "KeyError: 'Unable to open object (component not found)'" because the way that you structured the data saved in the h5 file (block 11 in your PDF) does not match what is expected when it is loaded.

I think that my last suggestion and the original path you were on might be an easier way to obtain results if you are simply interested in calculating an environmental contour. I would be curious to understand why the np.array suggestion didn't work. Ultimately, the data that you are sending in through the h5 file format is the same as the text data that you were having issues with previously, so even if you navigate the h5 file, you may get the same error when it comes to calculating the contour because the data is still not structured the way the code expects somehow.

[andreowhite]

@aubreyeckert I just found out why it did not work. It had omitted the third variable dateNum. I made this correction and it worked. See last page of the attached. trial3WDRT_code.pdf

However, I have 2 queries: 1) The x-axes still says Te and not T. Is it just form or is there real substance and Te is actually what was used. 2) Is there a way to get a csv of the Hs and T values for the contour or do I just pick from the graph.

Also, the contour does not seem to fit my data very well. My data entered is 3hr storms. However, the NBDC data has 1 hr storms.

Let me know what you think.

Many thanks.

AOAW

[aubreyeckert]

@andreowhite Great, glad it is working! Here are some answers to your questions. For more details and related questions, please refer to the code documentation and source code, as these answers are also included there:

1. The axis markings are hard-coded (the data is whatever you sent in to the calculation), so if you would like to produce plots with the correct markings, you can pull the plotting code out of the source code and change the markings.
2. The getContours() function returns the contour values (example from documentation: Hs_Return, T_Return = pca46022.getContours(Time_SS, Time_r,nb_steps)), so you can read these and save them as you need.
3. The sea state duration is a parameter that is input into the contour function (Time_SS) and in your case should be set to 3 based on the description above.

The provided analysis examples could also be helpful as you continue to develop and refine your analysis from here: http://wec-sim.github.io/WDRT/examples.html.

[andreowhite]

@aubreyeckert noted. Many thanks.

AOAW

[andreowhite]

Hi @aubreyeckert I am trying to do the following and would like to know if it can be done: Option 1 1) Find the max. daily significant wave heights 2) Fit the GEV to the max daily values 3) Plot the contours using max daily value Option 2 1) Find the peaks over a chosen thershold value, u 2)Fit the GP distrbution to the peaks over u 3) Plot the contours using the peaks over u

I started with option 1 but I seem to get an empty array for block_maxima. See attached

block_maxima.pdf

Note sure why this is so.

AOAW

[ssolson]

@andreowhite I hope all is well with you and your research. As noted earlier in this thread WDRT should create the contours for whatever data it is passed (i.e. it is a statistical method independent of the type of data assuming it is in the expected form).

If I was going attempt the type of data analysis you outline above I would convert the data to a pandas DataFrame and index by datetime. Then I would resample the data daily by max. This functionality can be found in the Pandas documentation (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.resample.html). If you are interested there is a method which does this being developed into MHKiT (https://github.com/MHKiT-Software/MHKiT-Python/pull/53).

At this time I have removed the reference to Python 2.7 from the documentation. Thank you for bringing this to our attention.

The functionality of the WDRT toolbox is outlined in the WDRT documentation. If you have a specific question about current WDRT functionality please open a new issue related to that functionality or documentation. Again thank you for your interest in WDRT.

[andreowhite]

Thank you.

Regards, AOAW

From: ssolson notifications@github.com Sent: 29 July 2020 20:41 To: WEC-Sim/WDRT WDRT@noreply.github.com Cc: Andre White andre.white@plymouth.ac.uk; Mention mention@noreply.github.com Subject: Re: [WEC-Sim/WDRT] Installing WDRT package into python (#45)

@andreowhitehttps://github.com/andreowhite I hope all is well with you and your research. As noted earlier in this thread WDRT should create the contours for whatever data it is passed (i.e. it is a statistical method independent of the type of data assuming it is in the expected form).

If I was going attempt the type of data analysis you outline above I would convert the data to a pandas DataFrame and index by datetime. Then I would resample the data daily by max. This functionality can be found in the Pandas documentation (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.resample.html). If you are interested there is a method which does this being developed into MHKiT (MHKiT-Software/MHKiT-Python#53https://github.com/MHKiT-Software/MHKiT-Python/pull/53).

At this time I have removed the reference to Python 2.7 from the documentation. Thank you for bringing this to our attention.

The functionality of the WDRT toolbox is outlined in the WDRT documentation. If you have a specific question about current WDRT functionality please open a new issue related to that functionality or documentation. Again thank you for your interest in WDRT.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/WEC-Sim/WDRT/issues/45#issuecomment-665863196, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALMLCWB34VOVQD3F2ORGOFLR6B3LTANCNFSM4OP2NRMA.

---

[http://www.plymouth.ac.uk/images/email_footer.gif]http://www.plymouth.ac.uk/worldclass

This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.