search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
725 stars 38 forks source link

[REVIEW]: Osier: A Python package for multi-objective energy system optimization #6919

Open editorialbot opened 5 months ago

editorialbot commented 5 months ago

Submitting author: !--author-handle-->@samgdotson<!--end-author-handle-- (Samuel Dotson) Repository: https://github.com/arfc/osier Branch with paper.md (empty if default branch): joss Version: v0.3.1 Editor: !--editor-->@prashjha<!--end-editor-- Reviewers: @fredshone, @victoraalves Archive: 10.5281/zenodo.14216170

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/183a04edba2d4952fa1e30c419a844b3"><img src="https://joss.theoj.org/papers/183a04edba2d4952fa1e30c419a844b3/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/183a04edba2d4952fa1e30c419a844b3/status.svg)](https://joss.theoj.org/papers/183a04edba2d4952fa1e30c419a844b3)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@fredshone & @victoraalves, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @prashjha know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @fredshone

📝 Checklist for @victoraalves

editorialbot commented 5 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 5 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.04 s (953.6 files/s, 197346.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          15            575            872           2155
Markdown                         9            113              0            420
Jupyter Notebook                 3              0           2332            316
TeX                              1             12              0            200
YAML                             3             12              8             84
DOS Batch                        1              8              1             26
make                             1              4              7              9
reStructuredText                 2             16             64              9
-------------------------------------------------------------------------------
SUM:                            35            740           3284           3219
-------------------------------------------------------------------------------

Commit count by author:

   286  Sam Dotson
     7  samgdotson
     2  Samuel Dotson
     1  yardasol
editorialbot commented 5 months ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s10479-015-1791-y is OK
- 10.1109/ACCESS.2020.2990567 is OK
- 10.1088/1748-9326/ab875d is OK
- 10.1007/s12532-011-0026-8 is OK
- 10.1016/j.erss.2021.101908 is OK
- 10.1287/inte.6.4.102 is OK
- 10.1016/j.ejor.2018.01.036 is OK
- 10.1016/j.rser.2014.02.003 is OK
- 10.1016/j.apenergy.2020.114728 is OK
- 10.1016/j.erss.2022.102913 is OK
- 10.3390/en12163046 is OK

MISSING DOIs

- No DOI given, and none found for title: 2023 Annual Technology Baseline (ATB)
- No DOI given, and none found for title: Openmod - Open Energy Modelling Initiative

INVALID DOIs

- None
editorialbot commented 5 months ago

Paper file info:

📄 Wordcount for paper.md is 1154

✅ The paper includes a Statement of need section

editorialbot commented 5 months ago

License info:

✅ License found: BSD 3-Clause "New" or "Revised" License (Valid open source OSI approved license)

editorialbot commented 5 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fredshone commented 5 months ago

Review checklist for @fredshone

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

fredshone commented 5 months ago

Really nice work well presented.

I require some changes detailed below with optional actions mixed in. A lot of these are based on compatability with Windows. Although not required I suggest both adding Windows to your CI and including env control in the docs. Mamba is very good. There are example projects here and here.

Installation and Installation Instructions:

On a Windows machine I hit two blockers; (i) python version (issue here), and (ii) solver installation documentation (issue here).

I strongly recommend adding a windows machine to the CI. Otherwise this is likely to reoccur.

I have also suggested that the default Windows solver be glpk. This is easy to install and will smooth CI. It is also more in keeping with JOSS (in my opinion). if you stick with a less open solver I think this needs to be explicitly mentioned in ther paper.

OPTIONAL, I also encourage use of mamba in the install instructions as per this issue.

Example usage: OPTIONAL - there is only a single example of multi-criteria and it is a little hidden at the end of a notebook. Please consider making this more obvious. At the moment multi-criteria appears as an after-thought rather than a core feature. OPTIONAL - please consider making the example notebooks more accessible in the project rather than just docs. See this issue.

Automated tests: Please add a code coverage report (eg pytest-cov).

State of the field and References: Please provide some more specific description and reference to similar tools. I think your focus on multi-criteria is very important, as you state - there are many other open energy system modelling tools. You would make the case better by more explicit comparison to similar projects that don't do multi-criteria. For example calliope is very similar in many ways but does not to multi-criteria without hacking the solver.

samgdotson commented 4 months ago

Hi @fredshone, I appreciate the review and the kind words. Addressing these issues is on my radar but I've hit a busy moment with other work -- I haven't forgotten about this, it's just on the back burner for now!

fredshone commented 4 months ago

Hi @fredshone, I appreciate the review and the kind words. Addressing these issues is on my radar but I've hit a busy moment with other work -- I haven't forgotten about this, it's just on the back burner for now!

No problem. You don't have to address everything at once - also happy to review bits and bobs when you have time. Similarly the review process is supposed to be open and two-way, some of my requests are quite specific, but you might be able to address them in better ways. Hope that makes sense.

prashjha commented 3 months ago

Hi @victoraalves, I hope you are doing well. I see you have not created a reviewer checklist yet. Could you please do so soon? If I can help you with anything, just let me know.

prashjha commented 3 months ago

Hi @victoraalves, could you please start the review? It has been two months since it started. I understand you are busy with other commitments. If you could provide some timeline, that would be helpful.

Best wishes, Prashant

victoraalves commented 3 months ago

Hi @prashjha firstly, sorry for the late reply. I will finalize my review by no later than next week. Again, my apologies.

victoraalves commented 3 months ago

Review checklist for @victoraalves

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

prashjha commented 3 months ago

Hi @prashjha firstly, sorry for the late reply. I will finalize my review by no later than next week. Again, my apologies.

Glad to hear from you and thank you for the update.

victoraalves commented 3 months ago

I have performed my initial review.

The tool is interesting and a good contribution! Congratulations, nice work.

I had some issues regarding Python version compatibility (already mentioned here by the other reviewer), as well as problems related to numpy 2.0 that breaks some of the functionality of the tool.

Installation

Installation on macOS was a bit problematic, due to the new Python versions and the Numpy version installed by default when doing "pip install". This needs to be clarified in the documentation and/or fixed in the project as well.

Testing

Some tests failed due to the lack of the "cbc" solver, despite the fact that I followed the installation instructions as recommended.

EDIT: Now I found out that this (and other solvers) need to be installed separately. My suggestion then is to have a conda-based installation of your package, which will help to automate all of the installation process.

Status of the field

As mentioned, I believe it's a good idea to highlight other tools that have the same (or similar) objectives and explain how Osier differentiates itself.

samgdotson commented 2 months ago

@fredshone @victoraalves

Thank you both for your reviews! @fredshone, thank you for making a pull request, as well.

Regarding the status of the field, how much elaboration are you both looking for? The paper is already ~1000 words. I have an open pull request that adds a direct reference to two open source packages. Is more elaboration necessary or will this suffice?

Two well known open-source ESOMs, Calliope [@pfenninger:2018] and Python for Power Systems Analysis (PyPSA) [@brown:2018], partially address equity issues by implementing MGA, but this does not resolve the limitation of mono-objective optimization.

Please let me know!

fredshone commented 2 months ago

@samgdotson short is good I agree. (from memory) I'd like to see specific mention on how your work builds on above by adding multi-objective functionality. This could be a one-liner.

FYI, I spoke to calliope authors and they agreed MO was not a proper or intended functionality. So lean on it hard I suggest.

More pressing for me personally is that when I went through the examples - the (single) multi-criteria example was at the end of a long notebook. I'd like to see it (perhaps with an even more minimal example) brought more front and center so that a new user (me in this case) can more quickly use the examples to understand the project scope.

fredshone commented 2 months ago

Nice job with the CI and operating systems btw 🥇

victoraalves commented 2 months ago

@samgdotson Thank you for addressing this. I agree that you don't have to craft a long discussion on this. To me, your current PR is sufficient.

samgdotson commented 1 month ago

@fredshone @victoraalves I pushed a final PR that should address the remaining comments.

This PR

If you both agree and are satisfied, what are the next steps? Thanks for your reviews!

fredshone commented 1 month ago

looks good, i will make a final skim through everything once merged.

samgdotson commented 1 month ago

@fredshone, all merged!

fredshone commented 1 month ago

@editorialbot generate pdf

fredshone commented 1 month ago

@editorialbot commands

editorialbot commented 1 month ago

Hello @fredshone, here are the things you can ask me to do:


# List all available commands
@editorialbot commands

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for branch
@editorialbot set joss-paper as branch

# Run checks and provide information on the repository and the paper file
@editorialbot check repository

# Check the references of the paper for missing DOIs
@editorialbot check references

# Generates the pdf paper
@editorialbot generate pdf

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Get a link to the complete list of reviewers
@editorialbot list reviewers
editorialbot commented 1 month ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fredshone commented 1 month ago

@editorialbot check references

editorialbot commented 1 month ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.1007/s10479-015-1791-y is OK
- 10.1109/ACCESS.2020.2990567 is OK
- 10.1088/1748-9326/ab875d is OK
- 10.1007/s12532-011-0026-8 is OK
- 10.1016/j.erss.2021.101908 is OK
- 10.1287/inte.6.4.102 is OK
- 10.1016/j.ejor.2018.01.036 is OK
- 10.1016/j.rser.2014.02.003 is OK
- 10.1016/j.apenergy.2020.114728 is OK
- 10.1016/j.erss.2022.102913 is OK
- 10.3390/en12163046 is OK
- 10.5334/jors.188 is OK
- 10.21105/joss.00825 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: 2023 Annual Technology Baseline (ATB)
- No DOI given, and none found for title: Openmod - Open Energy Modelling Initiative

❌ MISSING DOIs

- None

❌ INVALID DOIs

- None
fredshone commented 1 month ago

@samgdotson @prashjha - Looks ready to me, nice job 🙇🏻 .

prashjha commented 1 month ago

@fredshone, thank you for letting us know.

prashjha commented 1 month ago

Hi @victoraalves, I'm just checking to see if you are close to finishing the reviews! Thank you!

victoraalves commented 1 month ago

HI @prashjha LGTM, nice work!

samgdotson commented 3 weeks ago

@prashjha what are the next steps?

prashjha commented 3 days ago

Hi @fredshone and @victoraalves, thank you for your efforts in reviewing this submission!

prashjha commented 3 days ago

Hi @samgdotson, sorry for the delay.

I am reading the draft and will let you know if I have any suggestions. Next step would be recommend accept and one of the EiC will make the final call. It should be fairly quick.

Meanwhile, could you please archive (if not done already) the release using zenodo and provide the archive reference so that I can associate it with your JOSS submission? Also, please ensure that the zenodo archive's title matches the title of this JOSS article.

If you have updated a version of your code, let me know, and I can update it here.

prashjha commented 3 days ago

@editorialbot generate pdf

prashjha commented 3 days ago

@editorialbot generate pdf

editorialbot commented 3 days ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 3 days ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

prashjha commented 3 days ago

Hello @samgdotson, I read the article and it looks good to me. Please read my comments above and respond.

samgdotson commented 3 days ago

Hi @prashjha, here is the Zenodo publication is available here: 10.5281/zenodo.14216170

DOI

The current version of the code is v0.3.1, thanks!

Let me know if I need to do anything else.

prashjha commented 2 days ago

@editorialbot set 10.5281/zenodo.14216170 as archive

editorialbot commented 2 days ago

Done! archive is now 10.5281/zenodo.14216170

prashjha commented 2 days ago

@editorialbot set v0.3.1 as version

editorialbot commented 2 days ago

Done! version is now v0.3.1

prashjha commented 2 days ago

Thanks, @samgdotson!

prashjha commented 2 days ago

@editorialbot recommend-accept

editorialbot commented 2 days ago 
Attempting dry run of processing paper acceptance...
editorialbot commented 2 days ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.1007/s10479-015-1791-y is OK
- 10.1109/ACCESS.2020.2990567 is OK
- 10.1088/1748-9326/ab875d is OK
- 10.1007/s12532-011-0026-8 is OK
- 10.1016/j.erss.2021.101908 is OK
- 10.1287/inte.6.4.102 is OK
- 10.1016/j.ejor.2018.01.036 is OK
- 10.1016/j.rser.2014.02.003 is OK
- 10.1016/j.apenergy.2020.114728 is OK
- 10.1016/j.erss.2022.102913 is OK
- 10.3390/en12163046 is OK
- 10.5334/jors.188 is OK
- 10.21105/joss.00825 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: 2023 Annual Technology Baseline (ATB)
- No DOI given, and none found for title: Openmod - Open Energy Modelling Initiative

❌ MISSING DOIs

- None

❌ INVALID DOIs

- None