search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
694 stars 36 forks source link

[REVIEW]: ETHOS.FINE: A Framework for Integrated Energy System Assessment #6274

Open editorialbot opened 5 months ago

editorialbot commented 5 months ago

Submitting author: !--author-handle-->@t-gross<!--end-author-handle-- (Theresa Gross) Repository: https://github.com/FZJ-IEK3-VSA/FINE Branch with paper.md (empty if default branch): paper Version: v2.3.0 Editor: !--editor-->@arfon<!--end-editor-- Reviewers: @GregorBecker, @olejandro, @leonardgoeke Archive: Pending

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/c4c62e14f2273e97606b1eb96d58bec6"><img src="https://joss.theoj.org/papers/c4c62e14f2273e97606b1eb96d58bec6/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/c4c62e14f2273e97606b1eb96d58bec6/status.svg)](https://joss.theoj.org/papers/c4c62e14f2273e97606b1eb96d58bec6)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@GregorBecker & @olejandro & @leonardgoeke, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @arfon know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @GregorBecker

📝 Checklist for @leonardgoeke

📝 Checklist for @olejandro

editorialbot commented 5 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 5 months ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=0.39 s (406.1 files/s, 143796.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          92           4996           8228          24874
Jupyter Notebook                12              0          12442           2506
HTML                             2            165              4            605
TeX                              1             31              0            482
reStructuredText                34            317            265            378
YAML                             6             21             37            205
Markdown                         5             70              0            167
DOS Batch                        3             21              1             46
TOML                             1              5              8             38
make                             1              4              7             10
Dockerfile                       1              5              1              8
-------------------------------------------------------------------------------
SUM:                           158           5635          20993          29319
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 5 months ago

Wordcount for paper.md is 1532

editorialbot commented 5 months ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.ijhydene.2019.08.032 is OK
- 10.1016/j.ijhydene.2020.12.197 is OK
- 10.3390/en12142825 is OK
- 10.1016/j.energy.2018.05.059 is OK
- 10.1016/j.ijhydene.2018.11.194 is OK
- 10.1007/s12532-011-0026-8 is OK
- 10.3390/en13030641 is OK
- 10.1016/j.apenergy.2021.117825 is OK
- 10.1016/j.apenergy.2022.119029 is OK
- 10.1016/j.ijhydene.2021.04.036 is OK
- 10.1016/j.apenergy.2018.01.023 is OK
- 10.1016/j.rser.2018.11.020 is OK
- 10.1016/j.enbuild.2022.112480 is OK
- 10.1016/j.ijhydene.2021.04.036 is OK
- 10.3390/en15103801 is OK
- 10.3390/en15249517 is OK
- 10.1007/s11590-021-01826-w is OK
- 10.1016/j.adapen.2021.100063 is OK
- 10.1016/j.ijhydene.2023.05.308 is OK
- 10.1016/j.adapen.2023.100148 is OK
- 10.1016/j.ijhydene.2023.05.012 is OK
- 10.1016/j.rset.2023.100059 is OK
- 10.18154/RWTH-2023-01485 is OK
- 10.18154/RWTH-2023-01465 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 5 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

arfon commented 5 months ago

@GregorBecker, @olejandro, @leonardgoeke – This is the review thread for the paper. All of our communications will happen here from now on.

Please read the "Reviewer instructions & questions" in the first comment above. Please create your checklist typing: 

@editorialbot generate my checklist

As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention https://github.com/openjournals/joss-reviews/issues/6274 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule.

Also, noting that multiple reviewers here said they would not be able to complete their reviews until towards the middle-end of February.

arfon commented 5 months ago

:wave: @GregorBecker, @olejandro, @leonardgoeke – just checking in here to see if there's anything you need from me at this stage?

GregorBecker commented 5 months ago

Good morning @arfon, at the moment I don't need any further help from you, as soon as I find the time to review I will open my checklist. Kind Regards Gregor

GregorBecker commented 4 months ago

Review checklist for @GregorBecker

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

leonardgoeke commented 4 months ago

Review checklist for @leonardgoeke

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

leonardgoeke commented 4 months ago

Overall, the introduction of ETHOS.FINE is clear, comprehensible, and relevant. I have a few points listed below, following the guidelines above.

Functionality

The installation guide was clear, but mamba as a package installer was new to me and something I had to install separately. Therefore, it is worth briefly mentioning and explaining this choice in the paper. The long list of peer-reviewed publications applying the tool proves its functionality. Although this could be interesting in this context, there are no claims regarding performance.

General checks

All good here.

Documentation

The repository names the package's dependencies but only specifies the version in one case. In my experience with Python, packages are often not compatible with the newest version of a dependency. As a result, the user must invest a lot of time reconstructing a functioning configuration if the dependencies do not specify a version.

The documentation shows how to cite the package but gives no details on contributing or reporting problems.

Software Paper

The current state-of-the-art and need for the software is not discussed in the paper. The one paper that is supposed to prove the need is outdated and the tool is not differentiated from others in the field (Calliope, GenX, PyPSA, AnyMOD, etc.).

Earlier, the paper mentioned complexity reduction as a key capability of FINE. Therefore, some indicative information on performance is sensible, for instance, how long it takes to generate and solve a model with a certain number of time-steps, regions, and technologies. In this context, I would also like to know if the tools use pyomo as a backend for historical reasons or if it has an advantage over Linopy in the specific case.

olejandro commented 4 months ago

Review checklist for @olejandro

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

arfon commented 3 months ago

@t-gross – looks like there's a bunch of feedback from the reviewers here at this point. Have you started working on that already? If not, perhaps you could and respond here?

t-gross commented 3 months ago

I apologize for my late response. First, thank you very much for all the feedback! We have started to implement it but it will still take some time.

arfon commented 1 month ago

@t-gross – just checking in here on your updates?

t-gross commented 1 month ago

@t-gross – just checking in here on your updates?

Unfortunately, it will still take some time to get everything updated. We implemented some improvements to the code and readme. I hope to finish everything by end of June.