search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
694 stars 36 forks source link

[REVIEW]: Choco-solver: A Java library for constraint programming #4708

Closed editorialbot closed 1 year ago

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@cprudhom<!--end-author-handle-- (Charles Prud'homme) Repository: https://github.com/chocoteam/choco-solver Branch with paper.md (empty if default branch): Version: v4.10.10 Editor: !--editor-->@jbytecode<!--end-editor-- Reviewers: @skadio, @ozgurakgun Archive: 10.5281/zenodo.7185962

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/c4e6329e6ef023c0b856ad6722ec8e34"><img src="https://joss.theoj.org/papers/c4e6329e6ef023c0b856ad6722ec8e34/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/c4e6329e6ef023c0b856ad6722ec8e34/status.svg)](https://joss.theoj.org/papers/c4e6329e6ef023c0b856ad6722ec8e34)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@skadio & @ozgurakgun, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @jbytecode know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @skadio

📝 Checklist for @ozgurakgun

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 year ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=2.51 s (535.3 files/s, 93610.8 lines/s)
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
Java                           1242          22430          50553         149951
reStructuredText                 24            813           1263            966
Python                            7            212            325            962
Markdown                         17            366              0            957
Maven                             4             48             90            854
TeX                               2              9              0            775
Bourne Shell                     13            188            138            609
XML                               3             15             30            552
YAML                             12             60             81            423
ANTLR Grammar                     2             61             46            359
SVG                               4              0              0            302
DOS Batch                         1             36              2            243
make                              2             37              9            204
Jupyter Notebook                  3              0            568            190
Bourne Again Shell                2             29             64            101
HTML                              3             14              7             37
Dockerfile                        1             15              1             28
MUMPS                             1              0              0             21
Gradle                            1              0              0              7
--------------------------------------------------------------------------------
SUM:                           1344          24333          53177         157541
--------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 1 year ago

Wordcount for paper.md is 902

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/0004-3702(80)90051-X is OK
- 10.1016/0895-7177(93)90068-A is OK
- 10.1007/BF00137870 is OK
- 10.1007/3-540-49481-2_30 is OK
- 10.1007/11564751_71 is OK
- 10.1007/11564751_18 is OK
- 10.1007/11493853_7 is OK
- 10.1145/1452044.1452046 is OK
- 10.1007/978-3-642-04244-7_54 is OK
- 10.1007/s10601-014-9166-6 is OK
- 10.1007/s10601-013-9151-5 is OK
- 10.1007/978-3-642-29828-8_15 is OK
- 10.1007/978-3-540-30201-8_41 is OK
- 10.1109/TCIAIG.2011.2159716 is OK
- 10.29007/b4dz is OK
- 10.1007/s10601-015-9223-9 is OK
- 10.1007/978-3-319-23219-5_2 is OK
- 10.1109/ictai.2017.00164 is OK
- 10.1109/ICTAI.2019.00019 is OK
- 10.4230/LIPIcs.CP.2021.9 is OK
- 10.1016/j.simpa.2021.100085 is OK
- 10.1007/978-3-030-78375-4_8 is OK
- 10.1007/978-3-319-44953-1_40 is OK
- 10.1109/SCC.2019.00017 is OK
- 10.1016/j.eswa.2021.116149 is OK
- 10.1007/s10601-021-09324-7 is OK
- 10.1007/978-3-031-08011-1_21 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jbytecode commented 1 year ago

Dear reviewers @skadio, @ozgurakgun

Here is the reviewing thread. Please type

@editorialbot generate my checklist

to generate your checklist. In these checklist, there will be 20 checkbox items for each reviewer. Whenever you complete the corresponding task, you can check off the box.

The review process is interactive, so you can interact with the authors, other reviewers, and the editor (me).

You can also open new issues in the target repo, send pull request, and contribute in other ways as you wish.

Please, do not hesitate to ask me anything, anytime..

Thank you in advance.

cprudhom commented 1 year ago

@skabio

The paper reads well for a CP researcher, but for JOSS community, I have a few suggestions that should be easy to make:

The Abstract goes into a CP Overview and the link between Choco and CP is kinda gets lost and/or not clear. This might be obvious for a CP person but not necessarily to a general reader. Some of the technical details here in abstract are also "too technical", in my opinion. For example, it mixes constraints and predicates (are they the same?).

I will try to make the CP Overview easier to understand for non CP person. I made the choice to introduce Choco in the 2nd section, but I understand that it is a bit late, right?

Re: Abstract. If I wear a general user hat, it is not clear that's the value proposition. We need to answer why should the user/reader be interested using it. Choco has many great applications, why not mention a few of these industrial domains as success stories for CP/Choco? Then, the user can achieve in their domains using Choco. That "promise/proposition" shall be added to the abstract.

I can try emphasize the versatility of the library (well, the CP in general) through success stories and highlight the salient features of Choco.

We need an example CP problem to make the scenario concrete for the user. Why not consider N-Queens or sth similar (and then we can build on the CP Model). You might even make 2 models; one abstract model and then one Java Choco implementation. What we would like to highlight here is that there is no gap between problem description and the implementation. That "declarative" nature is the strength of CP/Choco.

I didn't think I could go into that kind of detail, but it's an option.

The example can help bridge the definitions (constraints, domains, variables) and the Features/Functionality.

@ozgurakgun Would you prefer to comment on the current version or wait for a new version that takes these remarks into account?

skadio commented 1 year ago

@ozgurakgun to save time/effort for our authors here, would you mind taking a quick look at the write-up (it is 2 pages) and comment on whether you would agree/disagree with my points on the manuscript.

My main commentary is from the perspective of a general JOSS reader (think non-CP researcher) for 1) a quick rephrase of the abstract to make the value proposition of the tool (and hence CP) clearer and 2) an illustrative example to make the CP concepts and Choco functionality more concrete to raise appetite to adopt/try the tool (and CP technology)

These two comments are motivated by the great opportunity here to showcase Choco and CP to the general open-source community via JOSS.

We would not want @cprudhom to change the paper back and forth if you interpret it differently, so a quick directional comment from you can help save us time/energy.

jbytecode commented 1 year ago

Dear reviewers

Could you please generate your checklists before digging into any discussions?

You can generate your checklists by typing

@editorialbot generate my checklist

here.

Thank you in advance.

jbytecode commented 1 year ago

@skadio - I am totally agreed with the example issue, a minimal example especially an nqueen problem would be very helpful for those who read the paper.

skadio commented 1 year ago

Review checklist for @skadio

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

ozgurakgun commented 1 year ago

Hi @skadio and all

I have now read the text and I agree it would benefit from an example. I am afraid n-queens may be too small/trivial?

Also I am not sure how much detail you think the authors should include. Constructing a full example using Choco would take too much space and wouldn't be very interesting. Instead the authors may give some example use cases without going into too much detail? Some of these examples can be chosen from those that will be interesting to the software development community: things like dependency resolution[1], type inference[2], test case generation[3], etc.

[1] https://www.well-typed.com/blog/2008/08/solving-the-diamond-dependency-problem/ [2] https://www.microsoft.com/en-us/research/publication/type-inference-as-constraint-solving-how-ghcs-type-inference-engine-actually-works/ [3] https://link.springer.com/chapter/10.1007/978-3-642-12592-8_10

(I am sorry the first 2 links are to blog articles instead of peer-reviewed papers, but several papers in these areas exist)

In addition to this point about examples I think it would make sense to mention/cite CP modelling languages that can use Choco as a backend solver. This shows how versatile Choco can be: it can be used directly to build a model or it can be used as a black-box solver with models generated from higher-level tools.

ozgurakgun commented 1 year ago

Review checklist for @ozgurakgun

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

jbytecode commented 1 year ago

@cprudhom , @ozgurakgun - of course, the listed subjects are appropriate for an example, but whatever the example is, it is better to get it as small as possible in size. The main objective of providing an example in JOSS papers is to present the basic usage of the software.

And of course, the decision will be made by our reviewers.

Thank you in advance.

cprudhom commented 1 year ago

Noted. Are there any deadlines?

jbytecode commented 1 year ago

@cprudhom - Although there is not a rigid time limit, you can start your edits whenever a suggestion comes from our reviewers without waiting further requests.

jbytecode commented 1 year ago

Dear @skadio, @ozgurakgun

Could you please update your status? How is going your review?

Thank you in advance.

skadio commented 1 year ago

I am mostly done with Software related parts of the checklist. This is a quite mature software with great amount of documentation and straightforward installation --just pointing to a single self-contaied jar file was enough to run/verify the usage examples.

I am waiting for the updated Paper from our authors. cc: @cprudhom. We had some commentary on the write-up previously, and I can finalize the review upon updated manuscript.

jbytecode commented 1 year ago

@cprudhom - could you please update your status?

cprudhom commented 1 year ago

sorry I'm caught up in the university year, but it's in my TODOs

cprudhom commented 1 year ago

Dear all, First of all, thank you for your comments. I have revised the structure of the article. But, I made the choice not to add a specific example / code but to link to available tutorials, I hope this is not prohibitive. So, here is a revision of the paper.

Best

cprudhom commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

skadio commented 1 year ago

Thank you for the revised version that improves over the initial submission. I especially like the new Summary and showcasing practical industry applications.

While I still believe that a formal Definition of a CSP/COP and then illustrative an example to make that Definition concrete would be good for the general reader, I won't hold that against the paper.

I doubt any CP-er would have difficulty reading the paper (in fact, they should know/heard of Choco anyway) so the question comes down to: "would the paper score the opportunity for a non-CP/MP/SAT person be excited about the paradigm to follow the references/links to find Modeling Tutorial. I will follow your judgement on that, and happy with the current version.

Minor comments on the write-up:

skadio commented 1 year ago

Related to my Statemenf of Need comment: the review form has a question on "Do the authors describe how this software compares to other commonly-used packages?" Something to consider.

With these minor comments, the paper is good to go from my end. Thank you @cprudhom

jbytecode commented 1 year ago

Dear @cprudhom

Could you please revise your code and the paper in light of our reviewer's suggestions. Please ping us when you have done with them.

Thank you in advance.

jbytecode commented 1 year ago

@cprudhom - could you please update your status and inform us on how is your edits going?

cprudhom commented 1 year ago

I'm waiting for a paragraph from my co-author about industrial applications. If that does not seem relevant to you, I will post-pone it and do my best to consider reviews by the end of the week

ozgurakgun commented 1 year ago

Just to confirm, I have gone through the checklist and am generally happy with the text. Adding that paragraph about industrial applications will only help the paper, so assuming we are not in a rush I would vote for waiting a little bit. Thanks for doing the work for this @cprudhom - increasing Choco's (and hence CP's) visibility in the open-source software development community is definitely a good thing.

cprudhom commented 1 year ago

@skadio

Statement of Need: is Choco the only Java solver, if so, stating that would make the statement stronger. If not, pointing out the differences with existing ones would be scholarly.

Actually, there are (at least) two other mature constraint solvers in Java : JaCoP and ACE (ex-Abscon). Both of them are very good constraint solver and their performances are quite similar to the ones of Choco but I think that Choco is more user-oriented (documentation, tutorials, frequent releases, regression and performance tests, ...).

L29 typo the CP --> CP

👍

L37, filters ,removes, is this a typo? why is removes in comma and/or filter in italics

I will keep filters and remove italics.

Nitpicking comment on L27-28: "Like ILP/SAT, CP is a field of MP". I kindly disagree with that statement, and many would do too, I think. Viewing SAT as MP could be problematic since its purely logical, and most CP constraints are not even numerical expressions but semantic constraints. Why not repharse to sth like: CP is a declarative paradigm where the user states the problem and the computer solves it. It is hence related to other approaches such as SAT and MP but is distinguished from them with its high-level modeling language and expressiveness. Sth like that.. Hope that makes sense.

You are right, I will rephrase the two first sentences.

cprudhom commented 1 year ago

@ozgurakgun It will be added by the end of the week

skadio commented 1 year ago

Thank you @cprudhom. Appreciate the input and revisions.

It might be good for the general user to be aware of other Java solvers then, with commentary on how Choco adds value along with some constructive verbiage around on those differences. This would help people position this work with respect to the field, and more such options also prove the maturity/applicability/relevance of the field as opposed to just one single option.

Thanks!

cprudhom commented 1 year ago

@editorialbot generate pdf

cprudhom commented 1 year ago

I just pushed a new version of the article, with (mainly) a new paragraph on industrial applications. I also conclude the Features and Functionality paragraph with few words on other Java solvers (but to be clear, I'm not really comfortable with this). I though it was easier for me to write this here instead of Statement of need because it mentioned concepts/functionalities presented after.

Best regards

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

cprudhom commented 1 year ago

Thinking twice, I'm not satisfied with the comparison with the (two) other solvers. Even if I could have made it deeper, it certainly requires a fair comparison with other CSP solvers, Java or not, with a commercial licence or not. But, I believe this is out of the scope. What do you think?

skadio commented 1 year ago

I found the Community/Integration section quite useful and a nice addition. It will definitely be useful for readers to learn about other relevant technology such as Minizinc/XCSP etc. The quick commentary on other (java) solvers is fair/reasonable.

I agree that going through an entire listing of CP solvers/tools would be out of scope, but at the very least, covering the "java" part seems relevant to me, since that's an angle of the paper as well.

There is an item in JOSS Review Checklikst:

This new section checks that box in my mind.

skadio commented 1 year ago

@jbytecode The paper is good to go from my end in its latest version.

jbytecode commented 1 year ago

@skadio - thank you for your review and spending your time for us.

@ozgurakgun - is your review completed?

If all of the issues are completed, then I can start my editorial tasks.

ozgurakgun commented 1 year ago

I now had a chance to read the final version, thanks for the updates @cprudhom, the paper is good ti be published as is for me as well.

jbytecode commented 1 year ago

@ozgurakgun - thank you for finalizing you review and spending your precious time for JOSS.

jbytecode commented 1 year ago

@editorialbot check references

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/0004-3702(80)90051-X is OK
- 10.1016/0895-7177(93)90068-A is OK
- 10.1007/BF00137870 is OK
- 10.1007/3-540-49481-2_30 is OK
- 10.1007/11564751_71 is OK
- 10.1007/11564751_18 is OK
- 10.1007/11493853_7 is OK
- 10.1145/1452044.1452046 is OK
- 10.1007/978-3-642-04244-7_54 is OK
- 10.1007/s10601-014-9166-6 is OK
- 10.1007/s10601-013-9151-5 is OK
- 10.1007/978-3-642-29828-8_15 is OK
- 10.1007/978-3-540-30201-8_41 is OK
- 10.1109/TCIAIG.2011.2159716 is OK
- 10.29007/b4dz is OK
- 10.1007/s10601-015-9223-9 is OK
- 10.1007/978-3-319-23219-5_2 is OK
- 10.1109/ictai.2017.00164 is OK
- 10.1109/ICTAI.2019.00019 is OK
- 10.4230/LIPIcs.CP.2021.9 is OK
- 10.1016/j.simpa.2021.100085 is OK
- 10.1007/978-3-030-78375-4_8 is OK
- 10.1007/978-3-319-44953-1_40 is OK
- 10.1109/SCC.2019.00017 is OK
- 10.1016/j.eswa.2021.116149 is OK
- 10.1007/s10601-021-09324-7 is OK
- 10.1007/978-3-031-08011-1_21 is OK

MISSING DOIs

- None

INVALID DOIs

- None
jbytecode commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jbytecode commented 1 year ago

@cprudhom - I've just sent a pull request for the paper. Please review them. If you are agreed with them, please apply. Thank you in advance.

cprudhom commented 1 year ago

@jbytecode I accepted the PR, thank you. Last question: how easy is it the upgrade the paper in the future ? For instance, we are working on Python bindings for Choco, which would be interesting to indicate in the paper. Should we delay the current process or restart it when it is ready?

jbytecode commented 1 year ago

@cprudhom - published papers can not be updated. It would be subject of a new submission.

jbytecode commented 1 year ago

@editorialbot check references

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/0004-3702(80)90051-X is OK
- 10.1016/0895-7177(93)90068-A is OK
- 10.1007/BF00137870 is OK
- 10.1007/3-540-49481-2_30 is OK
- 10.1007/11564751_71 is OK
- 10.1007/11564751_18 is OK
- 10.1007/11493853_7 is OK
- 10.1145/1452044.1452046 is OK
- 10.1007/978-3-642-04244-7_54 is OK
- 10.1007/s10601-014-9166-6 is OK
- 10.1007/s10601-013-9151-5 is OK
- 10.1007/978-3-642-29828-8_15 is OK
- 10.1007/978-3-540-30201-8_41 is OK
- 10.1109/TCIAIG.2011.2159716 is OK
- 10.29007/b4dz is OK
- 10.1007/s10601-015-9223-9 is OK
- 10.1007/978-3-319-23219-5_2 is OK
- 10.1109/ictai.2017.00164 is OK
- 10.1109/ICTAI.2019.00019 is OK
- 10.4230/LIPIcs.CP.2021.9 is OK
- 10.1016/j.simpa.2021.100085 is OK
- 10.1007/978-3-030-78375-4_8 is OK
- 10.1007/978-3-319-44953-1_40 is OK
- 10.1109/SCC.2019.00017 is OK
- 10.1016/j.eswa.2021.116149 is OK
- 10.1007/s10601-021-09324-7 is OK
- 10.1007/978-3-031-08011-1_21 is OK

MISSING DOIs

- None

INVALID DOIs

- None