[REVIEW]: LoadProfileGenerator: An Agent-Based Behavior Simulation for Generating Residential Load Profiles

whedon commented 3 years ago

Submitting author: !--author-handle-->@noah80@fraukewiese<!--end-editor-- Reviewers: @mewilhel, @szkafander Archive: 10.5281/zenodo.6304843

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/919ba68d6f5c2b0a0f9ece86bb74a535"><img src="https://joss.theoj.org/papers/919ba68d6f5c2b0a0f9ece86bb74a535/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/919ba68d6f5c2b0a0f9ece86bb74a535/status.svg)](https://joss.theoj.org/papers/919ba68d6f5c2b0a0f9ece86bb74a535)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@szkafander & @mewilhel , please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @fraukewiese know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @mewilhel

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@noah80) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @szkafander

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@noah80) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @samuelduchesne , @mewilhel it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 3 years ago

Wordcount for paper.md is 1054

whedon commented 3 years ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=1.30 s (905.6 files/s, 163655.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C#                            1026          17958          25378         116832
XML                             39            245            304          26567
XAML                            75           1100           1186          20613
MSBuild script                  24            121              0            988
TeX                              1             26              0            265
Markdown                         3            108              0            236
DOS Batch                        4              3              1             40
JSON                             1              0              0              5
-------------------------------------------------------------------------------
SUM:                          1173          19561          26869         165546
-------------------------------------------------------------------------------

Statistical information for the repository '75fc54a29a6cebc3d081fd58' was
gathered on 2021/08/05.
No commited files with the specified extensions were found.

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.apenergy.2020.115784 is OK
- 10.3390/en12050956 is OK
- 10.3390/buildings10080138 is OK
- 10.1016/j.est.2019.100763 is OK
- 10.3390/data5040102 is OK
- 10.1016/j.renene.2020.07.153 is OK
- 10.24451/arbor.9325 is OK
- 10.24451/arbor.9321 is OK
- 10.24451/arbor.7561 is OK
- 10.1016/j.rser.2012.08.013 is OK
- 10.1016/j.enbuild.2016.04.069 is OK
- 10.1016/j.rser.2008.09.033 is OK
- 10.1016/j.apenergy.2015.12.089 is OK
- 10.1016/j.enbuild.2019.109667 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon commented 3 years ago

Failed to discover a valid open source license.

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fraukewiese commented 3 years ago

@samuelduchesne , @mewilhel – This is the review thread for the paper. All of our communications will happen here from now on.

Please read the "Reviewer instructions & questions" in the first comment above.Both reviewers have checklists at the top of this thread (in that first comment) with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention #3574 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule.

fraukewiese commented 3 years ago

@noah80 I see that you name the MIT-license in your readme but it is not recognized by Github, probably the license-file is missing - could you please check on that?

noah80 commented 3 years ago

Thanks for agreeing to the review!

I fixed the license issue just now. Thanks for pointing that out.

If I can help in any way with the review, let me know?

fraukewiese commented 3 years ago

Hi @samuelduchesne , @mewilhel : If you have any further questions on the review, please do not hesitate to ask :)

whedon commented 3 years ago

:wave: @samuelduchesne , please update us on how your review is going (this is an automated reminder).

whedon commented 3 years ago

:wave: @mewilhel , please update us on how your review is going (this is an automated reminder).

mewilhel commented 3 years ago

@fraukewiese I've looked over the software and paper and I'm ready to go through my checklist but I'm receiving the following message when I attempt to accept the invite: "Sorry, we couldn't find that repository invitation. It is possible that the invitation was revoked or that you are not logged into the invited account." What are good next steps to resolve this?

danielskatz commented 3 years ago

@whedon re-invite @mewilhel as reviewer

whedon commented 3 years ago

OK, the reviewer has been re-invited.

@mewilhel please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

fraukewiese commented 3 years ago

Thanks @danielskatz and sorry @mewilhel ! Hope that works now.

mewilhel commented 3 years ago

@fraukewiese Everything looks good on my end now. Thanks!

samuelduchesne commented 3 years ago

Sorry, having some difficulty getting my hands on a Windows computer to test out the tool.

fraukewiese commented 3 years ago

@samuelduchesne : Thanks for the Update! I can understand that this is difficult. Do you think you will find a Windows computer for the review?

samuelduchesne commented 3 years ago

@fraukewiese it looks like it will be a while before a get my hands on one. I am transitioning into a new job and had to return my lab computer. I don’t have admin rights on the new one.

I might be quicker to assign a reviewer?

fraukewiese commented 3 years ago

@samuelduchesne - that is a pity, your experience fits so well to the submission, but I can very well understand the problem. So good start into your new job, I will try to find another reviewer.

fraukewiese commented 3 years ago

@jackbrookes Would you be willing to review the submission LoadProfileGenerator: An Agent-Based Behavior Simulation for Generating Residential Load Profiles ? The review has started already, but another reviewer could not make it, so it would be great if you could review this submission, since your programming skills with C# fit perfectly and also the content might be of interest to you.

mewilhel commented 3 years ago

Alright, I’ve finished reviewing.

I had an initial issue when attempting to use the software based on system time zones. I was able to easily work around this. I’m an American reviewer and this package appear to be more targeted for an EU end-user so I don’t consider this issue to be gating but rather a chance for continuous improvement (error log text below).

Otherwise, everything loads well on Windows and Linux. The software addresses an existing gap and I found the functionality easy to use with the existing documentation. The paper is well-written and as such I recommend it for publication in its current state.

errorlog.txt An error occurred. The error message is: An exception was thrown while activating λ:CalculationEngine.Helper.DayLightStatus. The time zone set in Windows is not compatible with the time zone of the geographic location. Yes, this is stupid, but it's a bug in the library used to calculate the sunrise times. The LPG version is 10.5.0.59. The error was logged to the file errorlog.txt. Please send that file to the author of the program to support further development.

fraukewiese commented 3 years ago

@mewilhel : Thank you very much for your thorough review! @noah80 : Could you comment on the remark from mewilhel about the system time zones and the target end-users?

fraukewiese commented 3 years ago

For information: I am currently contacting potential second reviewer per mail.

noah80 commented 3 years ago

Thank you very much for the kind review!

The time zone error message only shows up if the time zone of the computer doesn't match the time zone you want to simulate. So if you want to select a german household, you need to set your computer to the german time zone. This is usually not an issue, since most people need load profiles for their home country. It's a bug in the library, as the error message says.

The sunrise/sunset-calculations can be used to determine when people turn on the light, but tests showed that using measured global solar irradiation data works better anyway. People tend to turn on the light in the winter for example if it is too dark even if the sun is up. So, the entire functionality is a bit obsolete and will eventually be removed.

fraukewiese commented 3 years ago

@szkafander @sptennak Would one of you be willing to review the submission LoadProfileGenerator: An Agent-Based Behavior Simulation for Generating Residential Load Profiles ? The review has started already, but another reviewer could not make it, so it would be great if you could review this submission, since your programming skills with C# fit perfectly and also the content might be of interest to you.

fraukewiese commented 3 years ago

@noah80 : If you have any other suggestions for a second reviewer for your submission, please let me know.

szkafander commented 3 years ago

@fraukewiese Sorry, busy now. I can start reviewing this early next month. Let me know if that works.

fraukewiese commented 3 years ago

@szkafander Early next month is fine. Thank you very much! So, I will assign you as 2nd reviewer.

fraukewiese commented 3 years ago

@whedon add @szkafander as reviewer

whedon commented 3 years ago

OK, @szkafander is now a reviewer

fraukewiese commented 3 years ago

@szkafander : Thank you very much for agreeing to review!

Please read the "Reviewer instructions & questions" in the first comment above.Both reviewers have checklists at the top of this thread (in that first comment) with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention #3574 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

fraukewiese commented 3 years ago

/ooo September 26 until October 3

fraukewiese commented 3 years ago

@szkafander : If you have any questions regarding the review, do not hesitate to ask.

szkafander commented 3 years ago

@fraukewiese I am ready to start the review. I can't edit the checklist. You might need to resend the invitation.

fraukewiese commented 3 years ago

@whedon remove @samuelduchesne as reviewer

whedon commented 3 years ago

OK, @samuelduchesne is no longer a reviewer

fraukewiese commented 3 years ago

@whedon re-invite @szkafander as reviewer

whedon commented 3 years ago

The reviewer already has a pending invite.

@szkafander please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

fraukewiese commented 3 years ago

@szkafander Sorry for that, by clicking on the link above it should work, let me know if it still does not work.

szkafander commented 3 years ago

@fraukewiese Clicked it, I got "This invitation has expired.". Still can't edit. This happened to me before, not sure what I'm doing wrong...

fraukewiese commented 3 years ago

@szkafander Sorry for that, I will try to find out what is wrong...

arfon commented 3 years ago

Hrm, not sure what's going on here. @szkafander - could you try one more time accepting the invite here? https://github.com/openjournals/joss-reviews/invitations

szkafander commented 3 years ago

@arfon works now, thanks!

szkafander commented 3 years ago

@fraukewiese I finished the review.

LoadProfileGenerator is an awesome, extensive, incredibly detailed program. There is significant academic effort behind its development.

Using the website, installation of the binary is pain-free. Building from source also went smoothly.

The website has good information on usage. As far as I can tell, the code works as intended. The manual is available upon request.

I encountered the timezone issue too.

Suggestions:

Add build instructions.
Make manual downloadable.
Paper, line 80: add Python wrapper location.
Allow changing the "Work" path (or make option more visible if it already exists). It is annoying that the program dumps its data onto the system drive.
Fix the timezone bug.

fraukewiese commented 3 years ago

@szkafander : Thank you very much for your review! And thanks for the patience with the non-working review-invite link.

fraukewiese commented 3 years ago

@noah80 : Please react/reply to the suggestions from szkafander.

noah80 commented 3 years ago

Thanks for the review!

Build instructions: Good idea. I'll add those.
Downloadable Manual: I'll add a link.
Python Wrapper: Good point. I'll add that.
Default Path: It's in the settings.
Time Zone Bug: Will be fixed soon.

I'll update this thread as soon as there is a new version. Hope that this is ok?

fraukewiese commented 3 years ago

@noah80 : Perfect. Just let us know when you are done with the update and improvements.

openjournals / joss-reviews