search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
712 stars 38 forks source link

[REVIEW]: elmada: Dynamic electricity carbon emission factors and prices for Europe #3625

Closed whedon closed 2 years ago

whedon commented 3 years ago

Submitting author: @mfleschutz (Markus Fleschutz) Repository: https://github.com/DrafProject/elmada Version: v0.1.0 Editor: @fraukewiese Reviewer: @nmstreethran, @olejandro Archive: 10.5281/zenodo.5566694

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/9a6bf04b8a7bbc6b29b2ad79103acf42"><img src="https://joss.theoj.org/papers/9a6bf04b8a7bbc6b29b2ad79103acf42/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/9a6bf04b8a7bbc6b29b2ad79103acf42/status.svg)](https://joss.theoj.org/papers/9a6bf04b8a7bbc6b29b2ad79103acf42)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@nmstreethran & @olejandro , please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @fraukewiese know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @nmstreethran

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @olejandro

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @nmstreethran, @olejandro it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 3 years ago

Failed to discover a Statement of need section in paper

whedon commented 3 years ago

Wordcount for paper.md is 480

whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.apenergy.2021.117040 is OK
- 10.5281/zenodo.5126771 is OK

MISSING DOIs

- None

INVALID DOIs

- None
whedon commented 3 years ago 
Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.21 s (246.4 files/s, 63359.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
SVG                              5              0             33           7892
Python                          32            922            423           2832
Markdown                         5            190              0            523
YAML                             4              9              4            120
HTML                             1              7              0             51
TeX                              1              5              0             45
JSON                             1              0              0             28
TOML                             1              1              1             18
INI                              1              0              2             10
-------------------------------------------------------------------------------
SUM:                            51           1134            463          11519
-------------------------------------------------------------------------------

Statistical information for the repository '66bc415bf7ea5b9ab9c9473b' was
gathered on 2021/08/18.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
flma0001                        47          6784           2607          100.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
flma0001                   4177           61.6          1.2                7.21
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fraukewiese commented 3 years ago

@olejandro , @nmstreethran – This is the review thread for the paper. All of our communications will happen here from now on.

Please read the "Reviewer instructions & questions" in the first comment above.

Both reviewers have checklists at the top of this thread (in that first comment) with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention #3625 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule.

fraukewiese commented 3 years ago

@mfleschutz : Please include a Statement of Need section in the paper of the submission

fraukewiese commented 3 years ago

@whedon generate pdf

mfleschutz commented 3 years ago

Thank you, @fraukewiese. I restructured and updated the paper.

fraukewiese commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fraukewiese commented 3 years ago

@nmstreethran, @olejandro : If you have any further questions regarding the review, do not hesitate to ask :)

whedon commented 3 years ago

:wave: @nmstreethran, please update us on how your review is going (this is an automated reminder).

whedon commented 3 years ago

:wave: @olejandro , please update us on how your review is going (this is an automated reminder).

olejandro commented 3 years ago

Thanks, @fraukewiese! :-)

mfleschutz commented 3 years ago

Thank you @nmstreethran for your valuable feedback in DrafProject/elmada/issues/1. I hope the testing options are clearer now.

fraukewiese commented 3 years ago

Sorry for mentioning this issue in another one, that was a mistake and I cannot delete it anymore, so please just ignore the HydDown: ... mentioning in this review here.

fraukewiese commented 3 years ago

@olejandro , please update us on how your review is going :)

olejandro commented 3 years ago

@mfleschutz, a few comments/questions from me on the paper:

olejandro commented 3 years ago

Another question @mfleschutz: for the functionality documentation, should one refer to the readme or is there another source?

mfleschutz commented 3 years ago

@olejandro, thank you for the valuable feedback.

  • In the statement of need you refer to environmental assessment of power demands. I would have used environmental assessment of electricity supply instead. Is there a particular reason for why you refer to demand rather than supply?
  • In the summary you state To properly assess this flexibility within energy system models, it is essential to know the wholesale price and associated carbon emissions per unit of electricity for a given time step. Could you clarify why it is essential and the type of energy system models you refer to? For example, in energy system models like Balmorel or TIMES, electricity price and and associated carbon emissions are the results of modelling.

I considered both points in the latest changes of the paper. Maybe some brief background: My modeling background lies in the optimization of the energy systems of individual industrial and commercial businesses. In such a low-level model, the intention is to evaluate the costs and emissions of electricity demand (see point 1) and the national electricity system is not modeled together with it, since only the prices and emission factors are of interest (see point 2).

Another question @mfleschutz: for the functionality documentation, should one refer to the readme or is there another source?

The software functionality is documented in the readme. The methodology and the data are described in the Applied Energy Paper. Higher-level code is documented using doc-strings in the code. Type hints are used throughout the project. An additional readthedocs-documentation was not seen as necessary due to the small number of high-level functions. It was omitted in favor of compact documentation so far. Of course, this might change as the project evolves. @olejandro, @nmstreethran, do you think it would add value to the project in the current state?

mfleschutz commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

nmstreethran commented 3 years ago

@mfleschutz I don't think a Read the Docs documentation is necessary at this point either. Since there are docstrings and hints available, the core functionality documented in the README is sufficient, in my opinion. It's definitely something to consider if you expand upon the functionality of the software in the future.

Apologies for the delay in completing the rest of my review. I'll get this done by tomorrow. Thanks!

mfleschutz commented 3 years ago

@olejandro, @nmstreethran, thank you for your feedback and time invest.

olejandro commented 2 years ago

Thank you @mfleschutz! The changes make it very clear now. Also the Applied Energy Paper (great work, btw!) is very useful in understanding the background. One more comment I have on the Statment of Need - for the available functionality in elmada, how much is it complementary rather than an alternative to e.g. electricityMap? Regarding the documentation, I find the current README sufficient. A few points that I've noticed:

olejandro commented 2 years ago

@mfleschutz, a very minor comment on the References section - for some reason DOI for elmada appears twice.

olejandro commented 2 years ago

@fraukewiese, need your guidance regarding A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?. The target audience is included in the summary, should @mfleschutz move/include it in the Statement of Need as well?

fraukewiese commented 2 years ago

@olejandro @mfleschutz : I agree that the target audience should be mentioned in the Statement of need. Furthermore, I agree to the comment from olejandro that some more reference to similar software/projects should be made in the paper explaining which similarities and which additional/different functionalities/features/aspects exist. ("One more comment I have on the Statment of Need - for the available functionality in elmada, how much is it complementary rather than an alternative to e.g. electricityMap?")

mfleschutz commented 2 years ago

Thank you, @olejandro and @fraukewiese, for your fruitful comments. I respond by topics:

Readme (get_residual_load and get_el_national_generation)

I added a table describing functions that provide pre-processed data, see this commit.

Geographic scope

Going through the question of geographic scope was especially insightful. With this commit, elmada now supports 30 European countries instead of the 20 originally analyzed in the APEN paper. In the APEN paper, countries were excluded due to missing data, space restrictions, or less interesting analyses e.g. for countries with only one conventional fuel type. Also, after the APEN publication, more data became available. So I made changes to the code and documentation to include the 10 additional countries to the elmada functionalities.

Paper

In this commit I expanded the Statement of Need section to elaborate, how elmada is a light no-cost alternative to commercial services with the focus on the modeling of decentral energy hubs. Also I removed the duplicate DOI.

mfleschutz commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

olejandro commented 2 years ago

Thank you, @mfleschutz. I don't have any further comments. @fraukewiese, I've completed my check list. Is there anything else I need to e. g. click?

fraukewiese commented 2 years ago

@olejandro Thank you very much for your thorough review!! Having finished the checklist means that you recommend the submission for publication, you do not have to do anything else, your job is finished here :)

fraukewiese commented 2 years ago

@nmstreethran : I see you're nearly finished with your review - thanks a lot ! Please let me know when you are done and what your recommendation is.

nmstreethran commented 2 years ago

Hi @mfleschutz @fraukewiese, I have completed my review and have a few minor comments, which you can find below. Overall, this is a very useful software package and I am happy to recommend to accept once these comments are addressed. I do apologise for the delay, and please let me know if any of my comments are unclear.

environment.yml

Paper

Documentation

mfleschutz commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mfleschutz commented 2 years ago

Thank you @nmstreethran for your comments, I considered them in the latest commit.

If there are no other comments, I would suggest releasing elmada v0.1.0 and add the new Zenodo DOI in the JOSS paper.

nmstreethran commented 2 years ago

Thanks @mfleschutz, I have no further comments.

fraukewiese commented 2 years ago

@nmsteethran: Thank you very much for your thorough work, testing, commenting, reviewing!

fraukewiese commented 2 years ago

@mfleschutz: In the paper, please check: Line 20: "elmada is developed in the open on GitHub". - "in the open" sounds strange to me, but I am not a native speaker, so just doublecheck, if this is a correct phrase. Line 65: remove the "and" in the sentence

fraukewiese commented 2 years ago

@mfleschutz At this point could you:

I can then move forward with accepting the submission.

mfleschutz commented 2 years ago

@mfleschutz: In the paper, please check: Line 20: "elmada is developed in the open on GitHub". - "in the open" sounds strange to me, but I am not a native speaker, so just doublecheck, if this is a correct phrase.

Thanks for the comment. I saw this actually many times. Some examples: https://developer.apple.com/swift, https://opensource.com/..., this JOSS paper.

mfleschutz commented 2 years ago

Thank you @fraukewiese, I just released elmada v0.1.0 🚀

The new zenodo DOI is 10.5281/zenodo.5566694

While checking the Zenodo metadata, I was wondering if Michael D. Murphy is to be included as an author. He contributed with (non-code) active project direction and is therefore co-author of the JOSS paper.

fraukewiese commented 2 years ago

@whedon set v0.1.0 as version

whedon commented 2 years ago

OK. v0.1.0 is the version.

fraukewiese commented 2 years ago

Congratulation to releasing v0.1.0 @mfleschutz ! Please make sure that the title and author list in the archive on zenodo is exactly matching the paper title and author list. If you want to make Michael D. Murphy co-author, please add his name also in the archive.

mfleschutz commented 2 years ago

Okay, thanks @fraukewiese. I updated the Zenodo metadata to match the paper title and author list. Also, I added the Zenodo DOI in the JOSS paper references and codemeta.json.