[REVIEW]: ModelingToolkit.jl: An Intermediate Representation for Scientific Domain-Specific Languages

whedon commented 5 years ago

Submitting author: !--author-handle-->@HarrisonGrodin@matbesancon<!--end-editor-- Reviewers: @clagms, @jpfairbanks Archive: Pending

Status

Status badge code:

HTML: <a href="https://submissions.juliacon.org/papers/55e99191cded1ab6fb0d3b1019f537cd"><img src="https://submissions.juliacon.org/papers/55e99191cded1ab6fb0d3b1019f537cd/status.svg"></a>
Markdown: [![status](https://submissions.juliacon.org/papers/55e99191cded1ab6fb0d3b1019f537cd/status.svg)](https://submissions.juliacon.org/papers/55e99191cded1ab6fb0d3b1019f537cd)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@clagms & @jpfairbanks, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/JuliaCon/proceedings-reviews/invitations

The reviewer guidelines are available here: https://proceedings.juliacon.org/guide/reviewers. Any questions/concerns please let @matbesancon know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @clagms

Conflict of interest

[x] As the reviewer I confirm that I have read the JuliaCon conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Authorship: Has the submitting author (@HarrisonGrodin) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation? Claudio created Issue #182
[x] Functionality: Have the functional claims of the software been confirmed? Claudio created Issue #184 and Issue #185
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is? Claudio created Issue #186
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified? Claudio created Issue #187
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support Claudio created Issue #187

Paper format

[x] Authors: Does the paper.tex file include a list of authors with their affiliations?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)? Claudio: No.
[x] Page limit: Is the page limit for full papers respected by the submitted document?

Content

[x] Context: is the scientific context motivating the work correctly presented?
[x] Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
[x] Results: are the results presented and compared to approaches with similar goals?

Review checklist for @jpfairbanks

Conflict of interest

[ ] As the reviewer I confirm that I have read the JuliaCon conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[ ] I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

[ ] Repository: Is the source code for this software available at the repository url?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[ ] Authorship: Has the submitting author (@HarrisonGrodin) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

[ ] Authors: Does the paper.tex file include a list of authors with their affiliations?
[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[ ] Page limit: Is the page limit for full papers respected by the submitted document?

Content

[ ] Context: is the scientific context motivating the work correctly presented?
[ ] Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
[ ] Results: are the results presented and compared to approaches with similar goals?

whedon commented 5 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @clagms, @jpfairbanks it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/JuliaCon/proceedings-review) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/JuliaCon/proceedings-review:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 5 years ago

Attempting PDF compilation. Reticulating splines etc...

whedon commented 5 years ago

PDF failed to compile for issue #52 with the following error:

/app/vendor/ruby-2.4.4/lib/ruby/2.4.0/find.rb:43:in block in find': No such file or directory - tmp/52 (Errno::ENOENT) from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/find.rb:43:incollect!' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/find.rb:43:in find' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-61fc68877343/lib/whedon/processor.rb:61:infind_paper_paths' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-61fc68877343/bin/whedon:50:in prepare' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/command.rb:27:inrun' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in invoke_command' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor.rb:387:indispatch' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/base.rb:466:in start' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-61fc68877343/bin/whedon:116:in<top (required)>' from /app/vendor/bundle/ruby/2.4.0/bin/whedon:23:in load' from /app/vendor/bundle/ruby/2.4.0/bin/whedon:23:in

'

matbesancon commented 5 years ago

@whedon generate pdf from branch paper

whedon commented 5 years ago

Attempting PDF compilation from custom branch paper. Reticulating splines etc...

whedon commented 5 years ago

:point_right: Check article proof :page_facing_up: :point_left:

matbesancon commented 5 years ago

all good to start the review here

clagms commented 5 years ago

Authors: Does the paper.tex file include a list of authors with their affiliations?

I cannot find the repository where the *.tex files are. The pdf satisfies this criteria though.

clagms commented 5 years ago

[ ] Page limit: Is the page limit for full papers respected by the submitted document?

Where is this limit defined? I cannot find the call for papers. Apologies for my ignorance :P

matbesancon commented 5 years ago

It's all on https://proceedings.juliacon.org/, reviewer and author guide. They respect the page limit, which is not too much of a big deal as long as it remains reasonable

clagms commented 5 years ago

The repository address: https://github.com/HarrisonGrodin/JuliaCon2019-Paper leads to a 404. I've updated the links to https://github.com/JuliaDiffEq/ModelingToolkit.jl but I think this was generated from the review process (so the problem is not really solved).

matbesancon commented 5 years ago

Right thanks, I updated the URL in another place where it needed fixing

clagms commented 5 years ago

I attach the pdf with the comments on the text: 10.21105.jcon.00052_review.pdf

matbesancon commented 4 years ago

ping @HarrisonGrodin can you let us know when you addressed the comments from the review? We should try to finish this soon

jpfairbanks commented 4 years ago

@harrisongrodin, Sorry for the delay in posting my notes.

I read the paper and think it should be accepted with minor revisions. I agree with most of @clagms comments.

Some broad considerations.

Why do scientific programmers use DSLs? I would say something about the difficulty and time consuming nature of correctly implementing scientific models as the complexity of the system grows.
Why not use the Expr type directly? As a user of MT, I can't really articulate it beyond "you want to use a different type because the usage is different".
You should discuss converting Operation to and from Expr, this is a common question for new users of MT and should be authoritatively presented in this paper.
What about sorted algebraic systems? Julia leverages type systems for scientific computing, shouldn't MT leverage sorts (the algebraic version on types)?

vchuravy commented 4 years ago

ping @HarrisonGrodin can you let us know when you addressed the comments from the review?

ChrisRackauckas commented 4 years ago

We're pulling this submission.

vchuravy commented 4 years ago

@whedon withdraw

whedon commented 4 years ago

Paper withdrawn.

JuliaCon / proceedings-review