[REVIEW]: JSOSuite.jl: Solving continuous optimization problems with JuliaSmoothOptimizers

editorialbot commented 3 months ago

Submitting author: !--author-handle-->@tmigot@odow<!--end-editor-- Reviewers: @sshin23, @mzagorowska Archive: 10.5281/zenodo.12588054

Status

Status badge code:

HTML: <a href="https://proceedings.juliacon.org/papers/80fccf1e96250bd37a5b25ec0ac38495"><img src="https://proceedings.juliacon.org/papers/80fccf1e96250bd37a5b25ec0ac38495/status.svg"></a>
Markdown: [![status](https://proceedings.juliacon.org/papers/80fccf1e96250bd37a5b25ec0ac38495/status.svg)](https://proceedings.juliacon.org/papers/80fccf1e96250bd37a5b25ec0ac38495)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@sshin23 & @mzagorowska, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @odow know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @sshin23

📝 Checklist for @mzagorowska

editorialbot commented 3 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper source files, type:

@editorialbot generate pdf

editorialbot commented 3 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s10107-004-0559-y is OK
- 10.5281/zenodo.3969045 is OK
- 10.5281/zenodo.2558627 is OK
- 10.21105/joss.05187 is OK
- 10.5281/zenodo.4605982 is OK
- 10.5281/zenodo.6940304 is OK
- 10.1137/141000671 is OK
- 10.1287/ijoc.2014.0623 is OK
- 10.5281/zenodo.3994983 is OK
- 10.5281/zenodo.2629034 is OK
- 10.1007/0-387-30065-1 is OK
- 10.5281/zenodo.2574162 is OK
- 10.1137/15M1020575 is OK
- 10.1287/mnsc.36.5.519 is OK
- 10.5281/zenodo.3700941 is OK
- 10.5281/zenodo.2655082 is OK
- 10.5281/zenodo.10434673 is OK
- 10.5281/zenodo.3991143 is OK
- 10.5281/zenodo.4758376 is OK
- 10.21105/joss.04736 is OK
- 10.1007/s101070100263 is OK
- 10.5281/zenodo.3948381 is OK
- 10.21105/joss.03991 is OK
- 10.1007/s10589-020-00201-2 is OK
- 10.1137/060654797 is OK
- 10.5281/zenodo.4309783 is OK
- 10.5281/zenodo.2559295 is OK
- 10.5281/zenodo.7153564 is OK
- 10.1007/s10589-014-9687-3 is OK
- 10.5281/zenodo.1188851 is OK
- 10.5281/zenodo.3672094 is OK
- 10.5281/zenodo.8174885 is OK
- 10.2139/ssrn.4601442 is OK

MISSING DOIs

- No DOI given, and none found for title: ReverseDiff.jl
- No DOI given, and none found for title: NLPModelsAlgencan.jl

INVALID DOIs

- None

editorialbot commented 3 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.03 s (1531.0 files/s, 204611.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
TeX                              8            274            200           2634
Julia                           17            142            230           1114
TOML                             7            181              1            899
Markdown                         9            199              0            621
YAML                             8              7              5            225
Ruby                             1              8              4             45
CSS                              1              5              0             22
-------------------------------------------------------------------------------
SUM:                            51            816            440           5560
-------------------------------------------------------------------------------

Commit count by author:

    68  tmigot
    21  RenanOD
    16  Abel Soares Siqueira
    15  github-actions[bot]
     7  JSOBot
     5  Tangi Migot
     4  Alexis Montoison
     3  Monssaf Toukal
     2  Abel Siqueira
     2  Dominique
     2  Renan Domingues
     1  Julia TagBot
     1  abelsiqueira
     1  geoffroyleconte

editorialbot commented 3 months ago

Paper file info:

📄 Wordcount for paper.tex is 1039

🔴 Failed to discover a Statement of need section in paper

editorialbot commented 3 months ago

License info:

✅ License found: Mozilla Public License 2.0 (Valid open source OSI approved license)

editorialbot commented 3 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

sshin23 commented 3 months ago

Review checklist for @sshin23

Conflict of interest

[x] I confirm that I have read the JuliaCon conflict of interest policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JCon for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/JuliaSmoothOptimizers/JSOSuite.jl?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@tmigot) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

[x] Authors: Does the paper.tex file include a list of authors with their affiliations?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Content

[x] Context: is the scientific context motivating the work correctly presented?
[x] Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
[x] Results: are the results presented and compared to approaches with similar goals?

sshin23 commented 3 months ago

Done reviewing. All the review criteria are met

odow commented 3 months ago

Thanks @sshin23!

@tmigot: You don't seem to cite ReverseDiff or Algencan in the paper? They can be removed from the .bib. I wonder if it also needs to be Statement of need 😆

I'm not sure why https://github.com/JuliaCon/proceedings-papers/blob/jcon.00161/jcon.00161/10.21105.jcon.00161.pdf

mzagorowska commented 3 months ago

Review checklist for @mzagorowska

Conflict of interest

[x] I confirm that I have read the JuliaCon conflict of interest policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JCon for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/JuliaSmoothOptimizers/JSOSuite.jl?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@tmigot) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

[x] Authors: Does the paper.tex file include a list of authors with their affiliations?
[ ] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Content

[x] Context: is the scientific context motivating the work correctly presented?
[x] Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
[ ] Results: are the results presented and compared to approaches with similar goals?

mzagorowska commented 3 months ago

I am done as well.

The code of conduct and conflict of interests links in the checklist don't work for me, so no ticks. Other missing ticks: I didn't find any comparison with other similar works in the paper. The actual documentation does contain benchmarking together with all the functionality descriptions and examples, but the link isn't in the paper. A suggestion - maybe include a brief description of the benchmarks & results in the paper, and provide a link to the full analysis?

odow commented 3 months ago

Hmm. The links should probably be:

@matbesancon @lucaferranti is this known?

lucaferranti commented 3 months ago

Good catch! Apparently the checklist template got the wrong links at some point. I opened a PR fixing it. Meanwhile, the links given by Oscar are good to refer too to tick the corresponding boxes

mzagorowska commented 3 months ago

Link-related ticks have been ticked.

tmigot commented 2 months ago

@editorialbot generate pdf

editorialbot commented 2 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

tmigot commented 2 months ago

Thank you @mzagorowska @sshin23 and @odow for the comments, and your patience.

@tmigot: You don't seem to cite ReverseDiff or Algencan in the paper? They can be removed from the .bib.

Thanks for noticing. I cleaned the bib file to stick to the ones cited in the paper.

Other missing ticks: I didn't find any comparison with other similar works in the paper. The [actual documentation ] (https://docs.juliahub.com/General/JSOSuite/0.1.1/)does contain [benchmarking ] (https://docs.juliahub.com/General/JSOSuite/0.1.1/benchmark/)together with all the functionality descriptions and >examples, but the link isn't in the paper. A suggestion - maybe include a brief description of the benchmarks & results >in the paper, and provide a link to the full analysis?

We added a couple of sentences regarding packages that in my opinion have a similar goal, Optimization.jl, and MathOptInterface.jl. JSOSuite and the JSO-compliant framework are very solver-oriented, and the role of this package is to simplify accessing the solvers being researched and developed here. Additionally, in my opinion, JSOSuite would simplify making a bridge with both Optimization.jl and MathOptInterface.jl (this one is partly done already).

We also added a sentence pointing out the documentation and tutorials. Sorry if my modifications are minimal, but I am also trying to stick to the 2-page limit for this extended abstract. If you still think it deserves further discussion in the paper, please let me know and I will try to prioritize.

I modified the paper in this pull request https://github.com/JuliaSmoothOptimizers/JSOSuite.jl/pull/173

odow commented 2 months ago

@editorialbot commands

editorialbot commented 2 months ago

Hello @odow, here are the things you can ask me to do:


# List all available commands
@editorialbot commands

# Add to this issue's reviewers list
@editorialbot add @username as reviewer

# Remove from this issue's reviewers list
@editorialbot remove @username from reviewers

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Assign a user as the editor of this submission
@editorialbot assign @username as editor

# Remove the editor assigned to this submission
@editorialbot remove editor

# Remind an author, a reviewer or the editor to return to a review after a 
# certain period of time (supported units days and weeks)
@editorialbot remind @reviewer in 2 weeks

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for version
@editorialbot set v1.0.0 as version

# Set a value for branch
@editorialbot set juliacon-paper as branch

# Set a value for repository
@editorialbot set https://github.com/organization/repo as repository

# Set a value for the archive DOI
@editorialbot set 10.5281/zenodo.6861996 as archive

# Mention the EiCs for the correct track
@editorialbot ping track-eic

# Run checks and provide information on the repository and the paper file
@editorialbot check repository

# Check the references of the paper for missing DOIs
@editorialbot check references

# Generates the pdf paper
@editorialbot generate pdf

# Recommends the submission for acceptance
@editorialbot recommend-accept

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Flag submission with questionable scope
@editorialbot query scope

# Get a link to the complete list of reviewers
@editorialbot list reviewers

# Creates a post-review checklist with editor and authors tasks
@editorialbot create post-review checklist

# Open the review issue
@editorialbot start review

odow commented 2 months ago

@editorialbot check references

editorialbot commented 2 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s10107-004-0559-y is OK
- 10.5281/zenodo.3969045 is OK
- 10.5281/zenodo.2558627 is OK
- 10.5281/zenodo.4605982 is OK
- 10.1137/141000671 is OK
- 10.1007/0-387-30065-1 is OK
- 10.1137/15M1020575 is OK
- 10.1287/mnsc.36.5.519 is OK
- 10.5281/zenodo.10434673 is OK
- 10.5281/zenodo.3991143 is OK
- 10.5281/zenodo.4758376 is OK
- 10.5281/zenodo.3948381 is OK
- 10.21105/joss.03991 is OK
- 10.1007/s10589-020-00201-2 is OK
- 10.5281/zenodo.4309783 is OK
- 10.5281/zenodo.7153564 is OK
- 10.2139/ssrn.4601442 is OK
- 10.1287/ijoc.2021.1067 is OK

MISSING DOIs

- None

INVALID DOIs

- None

odow commented 2 months ago

@editorialbot check repository

editorialbot commented 2 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.03 s (1495.3 files/s, 193884.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
TeX                              8            255            196           2454
Julia                           17            142            230           1114
TOML                             7            181              1            899
Markdown                         9            199              0            621
YAML                             8              7              5            225
Ruby                             1              8              4             45
CSS                              1              5              0             22
-------------------------------------------------------------------------------
SUM:                            51            797            436           5380
-------------------------------------------------------------------------------

Commit count by author:

    68  tmigot
    21  RenanOD
    16  Abel Soares Siqueira
    15  github-actions[bot]
     7  JSOBot
     7  Tangi Migot
     4  Alexis Montoison
     3  Monssaf Toukal
     2  Abel Siqueira
     2  Dominique
     2  Renan Domingues
     1  Julia TagBot
     1  abelsiqueira
     1  geoffroyleconte

editorialbot commented 2 months ago

Paper file info:

📄 Wordcount for paper.tex is 1064

🔴 Failed to discover a Statement of need section in paper

editorialbot commented 2 months ago

License info:

✅ License found: Mozilla Public License 2.0 (Valid open source OSI approved license)

odow commented 2 months ago

I opened a small tweak: https://github.com/JuliaSmoothOptimizers/JSOSuite.jl/pull/175

odow commented 2 months ago

@editorialbot check repository

editorialbot commented 2 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.03 s (1541.1 files/s, 199825.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
TeX                              8            255            196           2454
Julia                           17            142            230           1114
TOML                             7            181              1            899
Markdown                         9            199              0            621
YAML                             8              7              5            225
Ruby                             1              8              4             45
CSS                              1              5              0             22
-------------------------------------------------------------------------------
SUM:                            51            797            436           5380
-------------------------------------------------------------------------------

Commit count by author:

    68  tmigot
    21  RenanOD
    16  Abel Soares Siqueira
    15  github-actions[bot]
     7  JSOBot
     7  Tangi Migot
     4  Alexis Montoison
     3  Monssaf Toukal
     2  Abel Siqueira
     2  Dominique
     2  Renan Domingues
     1  Julia TagBot
     1  Oscar Dowson
     1  abelsiqueira
     1  geoffroyleconte

editorialbot commented 2 months ago

Paper file info:

📄 Wordcount for paper.tex is 1064

🔴 Failed to discover a Statement of need section in paper

editorialbot commented 2 months ago

License info:

✅ License found: Mozilla Public License 2.0 (Valid open source OSI approved license)

odow commented 2 months ago

@matbesancon @lucaferranti any ideas on the "statement of need" thing?

tmigot commented 1 month ago

Hi @odow ! I was wondering if there was any update here? Thanks in advance.

lucaferranti commented 1 month ago

@matbesancon @lucaferranti any ideas on the "statement of need" thing?

Sorry I missed the ping. We recently updated the guidelines https://github.com/JuliaCon/proceedings-review/pull/166 . The consensus was that as long as the statement of need is clearly conveyed in the paper, it's not needed to have a section verbatim with that name. We should probably also disable that automatic check

odow commented 1 month ago

@editorialbot recommend-accept

editorialbot commented 1 month ago

Paper is not ready for acceptance yet, the archive is missing

odow commented 1 month ago

@tmigot can you upload a version of JSO to https://zenodo.org to get a DOI please?

tmigot commented 3 weeks ago

@editorialbot commands

editorialbot commented 3 weeks ago

Hello @tmigot, here are the things you can ask me to do:


# List all available commands
@editorialbot commands

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for branch
@editorialbot set juliacon-paper as branch

# Run checks and provide information on the repository and the paper file
@editorialbot check repository

# Check the references of the paper for missing DOIs
@editorialbot check references

# Generates the pdf paper
@editorialbot generate pdf

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Get a link to the complete list of reviewers
@editorialbot list reviewers

tmigot commented 3 weeks ago

@tmigot can you upload a version of JSO to https://zenodo.org to get a DOI please?

It is already done, here: https://zenodo.org/records/12588054

odow commented 3 weeks ago

@editorialbot set 10.5281/zenodo.12588054 as archive

editorialbot commented 3 weeks ago

Done! archive is now 10.5281/zenodo.12588054

odow commented 3 weeks ago

@editorialbot recommend-accept

editorialbot commented 3 weeks ago

Attempting dry run of processing paper acceptance...

editorialbot commented 3 weeks ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.1007/s10107-004-0559-y is OK
- 10.5281/zenodo.3969045 is OK
- 10.5281/zenodo.2558627 is OK
- 10.5281/zenodo.4605982 is OK
- 10.1137/141000671 is OK
- 10.1007/0-387-30065-1 is OK
- 10.1137/15M1020575 is OK
- 10.1287/mnsc.36.5.519 is OK
- 10.5281/zenodo.10434673 is OK
- 10.5281/zenodo.3991143 is OK
- 10.5281/zenodo.4758376 is OK
- 10.5281/zenodo.3948381 is OK
- 10.21105/joss.03991 is OK
- 10.1007/s10589-020-00201-2 is OK
- 10.5281/zenodo.4309783 is OK
- 10.5281/zenodo.7153564 is OK
- 10.2139/ssrn.4601442 is OK
- 10.1287/ijoc.2021.1067 is OK

🟡 SKIP DOIs

- None

❌ MISSING DOIs

- None

❌ INVALID DOIs

- None

editorialbot commented 3 weeks ago

:wave: @JuliaCon/jcon-eics, this paper is ready to be accepted and published.

Check final proof :point_right::page_facing_up: Download article

If the paper PDF and the deposit XML files look good in https://github.com/JuliaCon/proceedings-papers/pull/98, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

lucaferranti commented 3 weeks ago

@editorialbot accept

editorialbot commented 3 weeks ago

Doing it live! Attempting automated processing of paper acceptance...

editorialbot commented 3 weeks ago

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JCON! 🚨🚨🚨

Here's what you must now do:

Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/JuliaCon/proceedings-papers/pull/99
Wait five minutes, then verify that the paper DOI resolves https://doi.org/10.21105/jcon.00161
If everything looks good, then close this review issue.
Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

editorialbot commented 3 weeks ago

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://proceedings.juliacon.org/papers/10.21105/jcon.00161/status.svg)](https://doi.org/10.21105/jcon.00161)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/jcon.00161">
  <img src="https://proceedings.juliacon.org/papers/10.21105/jcon.00161/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://proceedings.juliacon.org/papers/10.21105/jcon.00161/status.svg
   :target: https://doi.org/10.21105/jcon.00161

This is how it will look in your documentation:

We need your help!

JuliaCon Proceedings is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://forms.gle/yANMe4c9YmuW9MWcA
Making a small donation to support our running costs here: https://numfocus.salsalabs.org/donate-to-jcon

odow commented 3 weeks ago

Thanks all!

JuliaCon / proceedings-review