editorialbot commented 1 year ago

Submitting author: !--author-handle-->@kbonney@rkurchin<!--end-editor-- Reviewers: @FlorianK13, @AdamRJensen Archive: 10.5281/zenodo.10126530

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/6c3554c98b1771125613cff94241847c"><img src="https://joss.theoj.org/papers/6c3554c98b1771125613cff94241847c/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/6c3554c98b1771125613cff94241847c/status.svg)](https://joss.theoj.org/papers/6c3554c98b1771125613cff94241847c)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@FlorianK13 & @AdamRJensen, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @rkurchin know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @FlorianK13

📝 Checklist for @AdamRJensen

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 1 year ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.24 s (371.7 files/s, 177447.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
SVG                              9              5            178          13086
Python                          34           1608           2805           5483
Jupyter Notebook                 8              0          16502            753
reStructuredText                24            336            274            655
Markdown                         4             81              0            241
TeX                              2             23              0            241
YAML                             5             18             13            130
DOS Batch                        1              8              1             26
make                             1              4              7              9
CSS                              1              1              0              4
-------------------------------------------------------------------------------
SUM:                            89           2084          19780          20628
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 1 year ago

Wordcount for paper.md is 1227

editorialbot commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1109/ACCESS.2020.3039182 is OK
- 10.21105/joss.00884 is OK
- 10.1109/ACCESS.2020.3021577 is OK
- 10.3390/en15145104 is OK
- 10.3390/app12041872 is OK
- 10.3390/en15145085 is OK
- 10.25080/Majora-92bf1922-00a is OK
- 10.1109/PVSC43889.2021.9518439 is OK
- 10.5281/zenodo.3509134 is OK
- 10.1109/PVSC45281.2020.9301021 is OK
- 10.2172/1734479 is OK
- 10.21105/joss.03021 is OK
- 10.1109/MCSE.2007.55 is OK

MISSING DOIs

- 10.1163/1574-9347_dnp_e612900 may be a valid DOI for title: Keras

INVALID DOIs

- None

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

FlorianK13 commented 1 year ago

Review checklist for @FlorianK13

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/sandialabs/pvOps?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@klbonne) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

rkurchin commented 1 year ago

Hi @AdamRJensen, just a reminder to get your review started when you can!

And @kbonney (I just noticed your GH handle was wrong in the top comment here; I just fixed that), any updates on progress in response to the remaining issues raised by @FlorianK13?

kbonney commented 1 year ago

Hi @AdamRJensen, just a reminder to get your review started when you can!

And @kbonney (I just noticed your GH handle was wrong in the top comment here; I just fixed that), any updates on progress in response to the remaining issues raised by @FlorianK13?

We'll start clearing up those issues @rkurchin. I also have a question for you regarding the formatting of the JOSS document. We had difficulties getting our html-style table to format correctly in the document. Our temporary solution was to use a screen-shot of the properly formatted table, but ideally this would be generated by the JOSS document creation procedure from source. Who can I reach out to for troubleshooting support for this sort of issue?

Relevant: https://github.com/sandialabs/pvOps/issues/52

rkurchin commented 1 year ago

@kbonney I've reached out on the editorial team Slack for someone who can help with the table formatting!

rkurchin commented 1 year ago

ping @openjournals/dev

arfon commented 1 year ago

@kbonney – do you have a good summary of what the issue is exactly with the table formatting?

kbonney commented 1 year ago

Yes @arfon. Our table includes bulleted lists inside the entries of the last two columns. To get this formatting right in markdown we were using \
tags to force linebreaks, otherwise the bullet points run together. Using these tags renders how we want in standard MD+HTML, but when we run the code through the joss draft workflow the \
tags seem to be ignored. You can see this happen in the document generated here: https://github.com/kbonney/pvOps/actions/runs/6026505579.

rkurchin commented 1 year ago

@openjournals/dev and/or @arfon, any thoughts here?

AdamRJensen commented 1 year ago

Review checklist for @AdamRJensen

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/sandialabs/pvOps?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@kbonney) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

AdamRJensen commented 1 year ago

Review comments from Adam R. Jensen

Comments to the paper

[x] In the first paragraph of the Statement of Need it would be adviseable to also mention the pvanalytics package and how this fits in with the already mentioned packages.
[x] Also, referencing https://openpvtools.readthedocs.io for a list of open source PV packages is highly advisable
[x] It would be nice to give an example of what an operations and maintenance record could contain, e.g., inverter malfunction. Just to make it more concrete for users not too familiar with PV performance data.
[x] In the Credit author statement, it would be nice if you also added the Software category as appropriate.
[x] I'm missing a concrete example in the Statement of Need. Here's a hypothetical example "Failures in PV power plants are noted in O&M logs, which often are overlooked due to difficulty in parsing these files, however it's important to understand what causes plant failures, e.g., inverter breakdown, jumping goats. pvops contains functions to solve this bla bla bla"
[x] Add a caption to Table 1.
[ ] Include one image from one of the tutorials in the paper as a visual example (optional)

Comments to the code

AdamRJensen commented 1 year ago

Reviewer note

While I have a lot of insight into the PV world, in particular PV modeling and Python, I'm not exactly the core audience for this package as I typically do not inspect PV performance data.

I'm writing this as I feel like it may be worthwhile to have a person in the core audience review the tutorials? Perhaps @FlorianK13 you represent this group? Alternatively, I have a couple of people in mind that may be willing to do this within the time frame of the review.

I hope it's not inappropriate to bring this up, but please let me know if this is the case. I'm merely interested in delivering the best possible review and ensuring great FOSS in the solar energy domain ❤️ 🌤️

FlorianK13 commented 1 year ago

@AdamRJensen When you say core audience, you mean people that work with PV performance data? Then I'm also not part of the core audience - hence if you know someone who is willing to spend some time to go through the tutorial (maybe even with csv data from an own plant) that would be very beneficial. As shown for example in https://github.com/sandialabs/pvOps/blob/master/examples/tutorial_textmodule.ipynb the functionalities to adapt the software so that you can use your own data exist, but I cannot test them due to the lack of own data stemming from this field.

AdamRJensen commented 1 year ago

@williamhobbs would you be interested in reviewing the pvOps tutorials?

Alternatively, I'm convinced @kandersolar could provide a helpful review of the tutorials. It should be mentioned that Kevin is affiliated with the same institution as the authors, though a third pair of eyes would be appreciated nevertheless.

Note, I'm NOT suggesting adding additional official reviewers to the paper as @FlorianK13 and I have that covered, but I merely want to get some extra targeted feedback on the tutorials / practical applications.

kbonney commented 1 year ago

Thanks for identifying this need @AdamRJensen. We would certainly appreciate having someone with experience in PV performance and O&M data take a look.

Also, I just want to mention that we are planning on addressing the issues that have been building up on the repository. However it may take a week or two as our team is currently busy with some end of financial year deliverables.

williamhobbs commented 1 year ago

@AdamRJensen, I am interested in reviewing the tutorials. I've only briefly looked at the readme and docs, so some pointers on where to start and what to test out might be helpful. Also, would I just try things out and provide comments here, or is there a more organized way to go about it?

It is feasible that I could use some real maintenance logs, like in https://github.com/sandialabs/pvOps/blob/master/examples/tutorial_textmodule.ipynb, but I'd have to check on that.

As a very general note about the tutorials, I noticed that there are too many characters on some lines for me to view everything in my browser, e.g.:

AdamRJensen commented 1 year ago

@AdamRJensen, I am interested in reviewing the tutorials. I've only briefly looked at the readme and docs, so some pointers on where to start and what to test out might be helpful. Also, would I just try things out and provide comments here, or is there a more organized way to go about it?

I would start with the first listed tutorial and go one by one (that's how I imagine a potential user would interact with it). My feeling was that some of the tutorials lacked sufficient explanation of the individual steps and description of the overall purpose, but that could simply due to my lack of practical experience. Any feedback that you might have, small or large, would be highly appreciated and should go into individual issues in the pvOps GitHub repository. You can start each issue title with [JOSS review assistance].

It is feasible that I could use some real maintenance logs, like in https://github.com/sandialabs/pvOps/blob/master/examples/tutorial_textmodule.ipynb, but I'd have to check on that.

If possible, that would be ideal! But if not then that's also ok.

rkurchin commented 1 year ago

@williamhobbs, thanks! If you're interested in reviewing the software overall, let me know and I can "officially" add you so you're able to have your own checklist.

williamhobbs commented 1 year ago

I just submitted a few issues. I'll add more if I get time to run through more tutorials in detail, but hopefully some of my feedback is general enough to be applied too all of them as appropriate.

rkurchin commented 1 year ago

Checking in with reviewers @FlorianK13 and @AdamRJensen on the status of things here!

AdamRJensen commented 1 year ago

Checking in with reviewers @FlorianK13 and @AdamRJensen on the status of things here!

@rkurchin I've completed the review and am only waiting on the authors to address the issues and PRs in the package repository (see my issue list above).

FlorianK13 commented 1 year ago

Checking in with reviewers @FlorianK13 and @AdamRJensen on the status of things here!

There are several open issues in the pvOps repository that need to be adressed by the authors.

kbonney commented 1 year ago

@arfon, any updates on the formatting issue?

Yes @arfon. Our table includes bulleted lists inside the entries of the last two columns. To get this formatting right in markdown we were using
tags to force linebreaks, otherwise the bullet points run together. Using these tags renders how we want in standard MD+HTML, but when we run the code through the joss draft workflow the
tags seem to be ignored. You can see this happen in the document generated here: https://github.com/kbonney/pvOps/actions/runs/6026505579.

arfon commented 1 year ago

The JOSS document pipeline is limited in what it can support sorry. While I don't like it, I would suggest either:

1) embedding an image of the table. 2) Trying using a LaTeX table. 3) Restructuring your tables to be less complex.

rkurchin commented 1 year ago

Thanks, Arfon. @kbonney, hopefully that's workable. Any updates regarding other aspects raised by reviewers above?

kbonney commented 1 year ago

Thanks, Arfon. @kbonney, hopefully that's workable. Any updates regarding other aspects raised by reviewers above?

We are still in the process of working through the issues. A bulk of them have been addressed, but we still have some work to do.

kbonney commented 1 year ago

@rkurchin, all of the issues brought up by reviewers on our GitHub page have been cleared.

For full transparency, our team is currently discussing with a collaborator the role of one of our modules (pvops.iv) in the software. We may end up deciding to move these methods to a separate repository, so we would like to hold off wrapping up the review until this decision is made so we can reflect it in the publication. It should not take more than a couple weeks - hopefully this does not cause any serious inconvenience.

If there are any other bookkeeping things we can take care of in the mean time please let me know.

rkurchin commented 1 year ago

That's fine – I'll go ahead and mark this review as paused and you can let me know whenever you're ready to conclude things.

kbonney commented 1 year ago

Hi @rkurchin, thanks for your patience. We have concluded our internal discussions and are ready to conclude the review process. We decided to keep the IV module as is, so no major changes are coming. Please let me know what the next steps are.

AdamRJensen commented 12 months ago

Review comments from Adam R. Jensen

Comments to the paper

[ ] In the first paragraph of the Statement of Need it would be adviseable to also mention the pvanalytics package and how this fits in with the already mentioned packages.

[ ] Also, referencing https://openpvtools.readthedocs.io for a list of open source PV packages is highly advisable

[ ] It would be nice to give an example of what an operations and maintenance record could contain, e.g., inverter malfunction. Just to make it more concrete for users not too familiar with PV performance data.

[ ] In the Credit author statement, it would be nice if you also added the Software category as appropriate.

[ ] I'm missing a concrete example in the Statement of Need. Here's a hypothetical example "Failures in PV power plants are noted in O&M logs, which often are overlooked due to difficulty in parsing these files, however it's important to understand what causes plant failures, e.g., inverter breakdown, jumping goats. pvops contains functions to solve this bla bla bla"

[ ] Add a caption to Table 1.

[ ] Include one image from one of the tutorials in the paper as a visual example (optional)

@kbonney did you address these comments to the paper?

kbonney commented 12 months ago

@AdamRJensen, no I haven't. I was wondering why there weren't any comments on the text, but i just hadn't tracked them down yet. I'll address these next week, thanks.

kbonney commented 12 months ago

The comments from Adam have been addressed, @rkurchin.

AdamRJensen commented 12 months ago

@editorialbot generate pdf

editorialbot commented 12 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

AdamRJensen commented 12 months ago

@kbonney Can you check that the most recent proof contains the changes you mention?

kbonney commented 12 months ago

@AdamRJensen It looks up to date to me

AdamRJensen commented 12 months ago

@rkurchin Green light from me to go on with publishing.

FlorianK13 commented 12 months ago

@rkurchin Same for me, I think it can be published as it is

kbonney commented 12 months ago

@editorialbot generate pdf

kbonney commented 12 months ago

Just added a couple small punctuation tweaks. We are all set on our end too, @rkurchin.

editorialbot commented 12 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

rkurchin commented 11 months ago

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

[x] Double check authors and affiliations (including ORCIDs)
[x] Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper.
[x] Archive the release on Zenodo/figshare/etc and post the DOI here.
[x] Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.
[x] Make sure that the license listed for the archive is the same as the software license.

Editor Tasks Prior to Acceptance

[x] Read the text of the paper and offer comments/corrections (as either a list or a PR)
[x] Check the references in the paper for corrections (e.g. capitalization)
[x] Check that the archive title, author list, version tag, and the license are correct
[x] Set archive DOI with @editorialbot set <DOI here> as archive
[x] Set version with @editorialbot set <version here> as version
[x] Double check rendering of paper with @editorialbot generate pdf
[x] Specifically check the references with @editorialbot check references and ask author(s) to update as needed
[ ] Recommend acceptance with @editorialbot recommend-accept

rkurchin commented 11 months ago

Thanks everyone! Authors, I'll do an editorial pass over the manuscript and send any comments shortly. In the meantime, please note the author tasks above and report back here with the relevant info as you complete them.

rkurchin commented 11 months ago

Some small editorial comments:

line 29: remove comma after "open-source"
line 32: are timeseries and text records the only data types? If not, "i.e." should be "e.g." (similarly on the next line for completeness and consistency as the only indicators of quality)
Regarding formatting for table 1, perhaps add lines between each row to more clearly separate the entries in the last column? Also, there's an errant period in the bottom row of the last column
I'd suggest breaking out the numbered list on lines 53-62 onto separate lines to improve readability
line 64: remove comma after "capabilities"

rkurchin commented 11 months ago

Additionally, in the first, third, and fifth references, Python should be capitalized.

kbonney commented 11 months ago

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

[x] Double check authors and affiliations (including ORCIDs)

[x] Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper.

[x] Archive the release on Zenodo/figshare/etc and post the DOI here.

[x] Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.

[x] Make sure that the license listed for the archive is the same as the software license.

Editor Tasks Prior to Acceptance

[x] Read the text of the paper and offer comments/corrections (as either a list or a PR)

[x] Check the references in the paper for corrections (e.g. capitalization)

[ ] Check that the archive title, author list, version tag, and the license are correct

[ ] Set archive DOI with @editorialbot set <DOI here> as archive

[ ] Set version with @editorialbot set <version here> as version

[ ] Double check rendering of paper with @editorialbot generate pdf

[ ] Specifically check the references with @editorialbot check references and ask author(s) to update as needed

[ ] Recommend acceptance with @editorialbot recommend-accept

@rkurchin JOSS Tag: version 0.3.0 Zenodo DOI: 10.5281/zenodo.10126530

Will address your editorial comments shortly

openjournals / joss-reviews

[REVIEW]: pvOps: a Python package for empirical analysis of photovoltaic field data #5755

Status

Reviewer instructions & questions

Checklists

Review checklist for @FlorianK13

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @AdamRJensen

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review comments from Adam R. Jensen

Comments to the paper

Comments to the code

Reviewer note

Review comments from Adam R. Jensen

Comments to the paper

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Editor Tasks Prior to Acceptance

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Editor Tasks Prior to Acceptance