porting pvl_PVsyst_parameter_estimation.m to python along with all associated codes to allow for full functionality

[mattguttenberg]

Following the Contributing page, I am creating this issue with regard to contributing ported code that I have been working on that will make pvl_PVsyst_parameter_estimation, and all associated code to make it run, accessible to pvlib-python. The comprehensive list of functions that I have / will be porting is:

pvl_PVsyst_parameter_estimation pvl_singlediode est_single_diode_param Schumaker_QSpline calc_theta_phi_exact update_io_known_n update_rsh_fixed_pt pvl_lambertw

I have already finished and done some testing with the scripts that are bolded while the others are either in the process of being completed or have not been started yet.

[cwhanse]

Hi Matt,

Thanks for taking this on. Let me know how I can help. I will contribute a data set we can use for code testing, and the corresponding Matlab parameter values.

For your matlab version do you have the pvl_robustfit.m code? If not I can send that to you.

Cliff

From: mattguttenberg [mailto:notifications@github.com] Sent: Monday, August 01, 2016 2:55 PM To: pvlib/pvlib-python pvlib-python@noreply.github.com Subject: [EXTERNAL] [pvlib/pvlib-python] porting pvl_PVsyst_parameter_estimation.m to python along with all associated codes to allow for full functionality (#227)

Following the Contributing page, I am creating this issue with regard to contributing ported code that I have been working on that will make pvl_PVsyst_parameter_estimation, and all associated code to make it run, accessible to pvlib-python. The comprehensive list of functions that I have / will be porting is:

pvl_PVsyst_parameter_estimation pvl_singlediode est_single_diode_param Schumaker_QSpline calc_theta_phi_exact update_io_known_n update_rsh_fixed_pt pvl_lambertw

I have already finished and done some testing with the scripts that are bolded while the others are either in the process of being completed or have not been started yet.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/pvlib/pvlib-python/issues/227, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFJNL33cFxoL-g-TMkVVzUjzTd5ASPfIks5qbl04gaJpZM4JZ_Lx.

[mattguttenberg]

Hi Cliff,

My matlab version does not have pvl_robustfit.m, I believe the pvl_PVsyst_parameter_estimation.m script is still using the corresponding matlab function. The bolded codes above have already been added to my fork with the corresponding tests that I had done if you would like to check them out. The tests are by no means comprehensive and check to see that the matlab version and the python version I wrote have the same outputs for a few examples so a data set for testing is definitely appreciated and probably needed. I have also only recently begun coding in python so if you see any room for improvement in the scripts that are already written, please let me know as I would love to learn some tricks with python. Along those lines, let me know if there is something I need to do in git to let you see my fork, I am also pretty new to git as well.

Thanks, Matt

[cwhanse]

Here’s the update we’ll release as PVLib 1.32. pvl_robustfit is a drop in for robustfit to get around requiring Matlab’s statistics toolbox. It uses code from the python statsmodels package if you don’t have the matlab toolbox. The communication between python and matlab is clunky, reading and writing temporary csv files.

Cliff

From: mattguttenberg [mailto:notifications@github.com] Sent: Monday, August 01, 2016 4:38 PM To: pvlib/pvlib-python pvlib-python@noreply.github.com Cc: Hansen, Clifford W cwhanse@sandia.gov; Comment comment@noreply.github.com Subject: [EXTERNAL] Re: [pvlib/pvlib-python] porting pvl_PVsyst_parameter_estimation.m to python along with all associated codes to allow for full functionality (#227)

Hi Cliff,

My matlab version does not have pvl_robustfit.m, I believe the pvl_PVsyst_parameter_estimation.m script is still using the corresponding matlab function. The bolded codes above have already been added to my fork with the corresponding tests that I had done if you would like to check them out. The tests are by no means comprehensive and check to see that the matlab version and the python version I wrote have the same outputs for a few examples so a data set for testing is definitely appreciated and probably needed. I have also only recently begun coding in python so if you see any room for improvement in the scripts that are already written, please let me know as I would love to learn some tricks with python. Along those lines, let me know if there is something I need to do in git to let you see my fork, I am also pretty new to git as well.

Thanks, Matt

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/pvlib/pvlib-python/issues/227#issuecomment-236729194, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFJNL_OmXJhAvB9eOq9WCcBEUwypvQ--ks5qbnU-gaJpZM4JZ_Lx.

[mattguttenberg]

Hi Cliff,

Thanks for the response. Sorry if I am missing something but I don't see the attached update you are referring to. Can you resend the attachment or direct me to where it is since I am still pretty new to github?

Thanks, Matt

[mattguttenberg]

I just wanted to give an update here. I have finished porting over PVsyst_parameter_estimation. From the couple of tests I have run, it seems that the code generates coefficients that are very similar to those that are generated from MATLAB (on average about .01% differences). However, I would still very much appreciate the data set to further validate the code if you are still able to provide it Cliff. I'm also not sure what style changes should be made so that it fits better with the pvlib toolbox but the push request seems have passed all of the checks. Any feedback would be greatly appreciated.

[cwhanse]

I'm converting an IV curve data set from Matlab to python. Is there already an example file format which is being used for IV curve data? If not, then I'll make one, but I'm having some difficulty getting a format that will smoothly interface with the ported PVsyst_parameter_estimation code. In particular, PVsyst... wants a dict containing numpy arrays. json.dump doesn't know how to parse a numpy array, so we'd have to write a workaround.

[adriesse]

All the IV curve files I have seen over the years were text files with columns I and V, and most often additional related columns. This data was preceded, or sometimes followed by, various orderly metadata. In general computer generated and fairly easy to parse.

On 2016-08-17 3:32, cwhanse wrote:

I'm converting an IV curve data set from Matlab to python. Is there already an example file format which is being used for IV curve data? If not, then I'll make one, but I'm having some difficulty getting a format that will smoothly interface with the ported PVsyst_parameter_estimation code. In particular, PVsyst... wants a dict containing numpy arrays. json.dump doesn't know how to parse a numpy array, so we'd have to write a workaround.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pvlib/pvlib-python/issues/227#issuecomment-240289364, or mute the thread https://github.com/notifications/unsubscribe-auth/AIlYQwHcV67WQaub1jhau94RTgEHNts7ks5qgmSegaJpZM4JZ_Lx.

Photovoltaic Performance Labs Germany Emmy-Noether-Str. 2 79110 Freiburg Germany

+49-761-8973-5603 (Europe) +49-174-532-7677 (Mobile) +1-613-817-1652 (North America)

www.pvperformancelabs.com

[wholmgren]

I go on vacation for a few weeks and now everyone has something to contribute! I should do that more often.

Thanks @mattguttenberg for your work on this. I will look at the pull request closely later today.

@cwhanse have you considered using pandas from_csv to get a (dict-like) DataFrame? If you don't want to keep it in Pandas format then you can use the DataFrame's to_dict method or access the DataFrame's .values attribute. numpy also has csv readers/writers, but I don't think they're as easy to use.

[dacoex]

@wholmgren have a great time! Maybe I will finds some time to take invitation more serious.

[cwhanse]

I'm going to write a function to read/write IV curve data from/to text as a dict of numpy arrays. I hadn't considered a dataframe because IV curves don't necessarily have a meaningful time stamp. The environmental data should be bundled with each IV curve.

It makes sense to me to eventually move toward a class definition for an IV curve.

On a sidenote, it may be easy to read/write the pvlib-matlab structure, I'll look into scipy.io.xxx capabilities. But I don't think we want to have .mat be the pvlib-python standard for these data.

[wholmgren]

Would hdf5 work for matlab?

[adriesse]

A dataframe doesn't have to have a time stamp index. It could be voltage in this case, or just an integer counter. Sometimes I change the index on the fly.

I started out putting metadata (environmental data) into a dict, but then moved to using pandas Series. Quite similar really, but sorted.

[cwhanse]

I'd rather stick with .mat as the native PVLib format. Do you want to use hdf5 for pvlib-python?

[wholmgren]

Well, I like hdf5 on principle, but there's a lot to be said for being able to use the same data files in both projects.

I think the binary formats are nice because they make it easy to support things like multidimensional arrays.

[mikofski]

I plan to pick this up again but might not be until q2

[cwhanse]

Closed by #708