search

JuliaCon / proceedings-review

7 stars 1 forks source link

[REVIEW]: Modia3D: Modeling and Simulation of 3D-Systems in Julia #43

Closed whedon closed 4 years ago

whedon commented 5 years ago

Submitting author: !--author-handle-->@MartinOtter<!--end-author-handle-- (Martin Otter) Repository: https://github.com/ModiaSim/Modia3D.jl Branch with paper.md (empty if default branch): Version: Editor: !--editor-->@matbesancon<!--end-editor-- Reviewers: @Datseris, @dawbarton Archive:

Status

status

Status badge code:

HTML: <a href="https://submissions.juliacon.org/papers/16d9193656436b4d4f774f73ab34d8c5"><img src="https://submissions.juliacon.org/papers/16d9193656436b4d4f774f73ab34d8c5/status.svg"></a>
Markdown: [![status](https://submissions.juliacon.org/papers/16d9193656436b4d4f774f73ab34d8c5/status.svg)](https://submissions.juliacon.org/papers/16d9193656436b4d4f774f73ab34d8c5)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@Datseris, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/JuliaCon/proceedings-reviews/invitations

The reviewer guidelines are available here: https://proceedings.juliacon.org/guide/reviewers. Any questions/concerns please let @matbesancon know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @Datseris

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Paper format

Content

Review checklist for @dawbarton

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Paper format

Content

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

MartinOtter commented 4 years ago

The Paper was improved based on reviewers comments.

In the paper and documentation there is a lack of comparison with other software that achieve similar characteristics. The software should be put into context of other existing software, and also attempt to educate the potential user on why they should choose this software, instead of other. I find it hard to believe that Modia3D.jl is the only software that does what it does.

This is now described better in the introduction to explain the differences to other software and also giving links to the large number of Game engines and multibody programs. This description could still be improved, but then other parts of the paper have to be removed due to the limit of 6 pages.

In the start of paragraph 2 the authors "compare" Modia3D.jl with game engines, but this is not a true comparison. They only mention that they use game developing paradigms without going into much depth. Me, as a potential user, should I consider comparing your software with Unreal Engine or with DifferentialEquations.jl ?

It was tried to make this more clear. Again, due to limited space the description is short. Modia3D is not intented to be a game engine (but use some good ideas from them).

I think the authors assume knowledge of what Modia or Modelica is. The reader of the paper may not necessarily have this knowledge (I for example do not). I think the authors should try to describe Modia in a couple of sentences before using it in text.

We added a table of the Modia platform with a short description and added some examples for the intended applications.

Both the paper as well as the documentation lack a statement of need. This also ties with my first comment; the authors do not really put the work into context.

Added some explanations in the introduction.

There is no discussion about performance but I think the authors should consider providing one. Does this software match the performance of its competitors? Does it exceed it?

As stated in the conclusion at the end, we put effort in functionality and did not yet tune it for performance.

Furthermore, novel algorithms seem to have been developed and be in use for this software (the three papers associated with the software, e.g. "Collision Handling with Variable-Step Integrators (EOOLT 2017, December)". Do these approaches also offer performance benefits?

The documentation is severely lacking examples as well as instructions. The package features are listed in detail, but instructions of how to leverage them are nowhere to be seen. Specifically, the "getting started page" is simply installation instructions, while all the "examples" of the software are presented like so:

# Examples to demonstrate kinematic movement of assemblies
include("$(Modia3D.path)/examples/kinematics/Move_DoublePendulum.jl")
include("$(Modia3D.path)/examples/kinematics/Move_FourBar.jl")
include("$(Modia3D.path)/examples/kinematics/Move_FourBar2.jl")

This is not enough. At least one example must be expanded in detail and every single step of the example must be explained, also in detail. In fact, at the moment I cannot see how a user would actually use Modia3D.jl, given the absence of guidance. Please be aware that listing an example is not enough: you rely on me being able to deduce what a command does by seeing its output. This is not the proper way to learn, as I might deduce many things wrongly.

The README.md file of Modia3D and the documentation has been significantly improved (version 0.4.0 of Modia3D)

Although optional, I think the authors should strongly consider bundling this plotting library they use with their software. The current situation is a no-go for me.

Probably, you mean 3D animation (not plotting)? In the paper the description was added that another team is developing an open source 2D/3D web app (Modiater).

Similar to Datseris' review my main criticism of the paper is the lack of context. What other rigid body dynamics packages should this be compared with? (Open source or closed source - a few reference points would be helpful.) While game engine comparisons are interesting (particularly what can be learnt from the structure of the code), I doubt they are the most appropriate in terms of the scientific objectives. Would software packages such as Mambo (http://danko.mechanical.illinois.edu/Mambo/MamboHome.htm) be an appropriate comparison? (I don't have any experience of it myself.)

Yes, we agree, the description could be better. We improved the introduction. Note, there are hundreds of multibody programs (smaller like Mambo and large ones like SIMPACK). However, none of them models non-mechanical parts directly (such as electrical motors, or an air-conditioning system). We tried to explain shortly in the introduction of the paper, that Modia3D will be an integrated part in a larger context.

Further to the lack of context, there could be a bit more information about the intended use cases of the software. From my own experience I can infer what they are likely to be but making it a bit more explicit would be good.

Added some examples in the introduction.

There are also a number of unqualified assertions in the paper, for example "The advantages of this approach compared to a response calculation with impulses is that (a) simulation results are closer to reality and (b) it is easier to treat complex contact situations correctly."; references supporting these statements would be much appreciated.

Added some more explanations here.

The online documentation provided is a little sparse, but well above what I would consider to be the bare minimum. With the information provided, I'd be happy to have a go at creating my own models though I suspect I would end up digging through the source code to find out necessary details.

The remaining item I haven't checked off is the references: the guidelines above appear to indicate that DOIs are required for all relevant publications and as far as I can see there aren't any provided. Other than that, the references look appropriate.

Added DOIs, but the latex templates do not use them.

A final comment/question, more from scientific curiousity than anything else, is it possible to use other impact laws other than linear elasticity for the contacts? For example the nonlinear relationships provided by Stronge's Impact Mechanics (Cambridge Press, 2018).

The documentation was now significantly enhanced (Modia3D 0.4.0) and has nonlinear elastic contact response calculation

Datseris commented 4 years ago

Hello,

I've went through the new version of the paper which I consider vastly improved; I accept for publication.

matbesancon commented 4 years ago

ping @dawbarton for the last checks in the list

dawbarton commented 4 years ago

Done. Looks good to me!

matbesancon commented 4 years ago

Some quick things: Table 1: Julia packages of the Modia platform. Page one col 2, "Whereas" is a bit awkward at the beginning of a sentence, prefer a comma with the previous sentence.

On the other hand, multi-body programs are usually far from the flexible setup of games.

Why? The sentence would benefit from a bit more justification and context

matbesancon commented 4 years ago

Page 3 col 2, replace cheap and cheaply with "low-cost", low computational cost or similar

matbesancon commented 4 years ago

Page 4 equation (3)

julia> 2.125 - 70 * 0.03
0.02499999999999991

Would be worth it to mention the order of magnitude of the resisual, 0.03m*s^-1 (rounded up) can be considered non-zero in some contexts

MartinOtter commented 4 years ago

Some quick things: Table 1: Julia packages of the Modia platform. Page one col 2, "Whereas" is a bit awkward at the beginning of a sentence, prefer a comma with the previous sentence.

Fixed

On the other hand, multi-body programs are usually far from the flexible setup of games.

Why? The sentence would benefit from a bit more justification and context

Added a reference to section 2. Due to lack of space, did not add more.

For information: Multi-body programs usually have "body" objects and "joints" and then associate attributes with a body (e.g. "markers" on the body or "visual" information representing the body). Games have "coordinate systems" as main objects and associate attributes with a coordinate system, such as geometry, visualization, dynamics, collision properties, light, camera, sound, video, etc.). The latter allows a much more flexible definition.

MartinOtter commented 4 years ago

Page 3 col 2, replace cheap and cheaply with "low-cost", low computational cost or similar Fixed.

matbesancon commented 4 years ago

@whedon generate pdf from branch JuliaCon2019-paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch JuliaCon2019-paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

MartinOtter commented 4 years ago

Page 4 equation (3)

julia> 2.125 - 70 * 0.03
0.02499999999999991

Would be worth it to mention the order of magnitude of the resisual, 0.03m*s^-1 (rounded up) can be considered non-zero in some contexts

Made no change due to lack of space. The numbers of these equations have been just visually extracted from the plot window and it is just a sketch to explain the bend in the figure. The actual numbers with more digits give an exact zero.

MartinOtter commented 4 years ago

point_right Check article proof page_facing_up point_left

Looks fine (just DOIs still missing in the literature section)

AndreaNeumayr commented 4 years ago

@matbesancon I’m wondering, it seems that the review checklists are fine from both reviewers, but the status is still “under review”. Do we have to do something?

matbesancon commented 4 years ago

Very sorry about the delays, this paper looks good to accept

matbesancon commented 4 years ago

@whedon generate pdf from branch JuliaCon2019-paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch JuliaCon2019-paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

matbesancon commented 4 years ago

@whedon check refs

whedon commented 4 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands
matbesancon commented 4 years ago

@whedon check references

matbesancon commented 4 years ago

@whedon commands

whedon commented 4 years ago

Here are some things you can ask me to do:

# List all of Whedon's capabilities
@whedon commands

# Assign a GitHub user as the sole reviewer of this submission
@whedon assign @username as reviewer

# Add a GitHub user to the reviewers of this submission
@whedon add @username as reviewer

# Re-invite a reviewer (if they can't update checklists)
@whedon re-invite @username as reviewer

# Remove a GitHub user from the reviewers of this submission
@whedon remove @username as reviewer

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

# Change editorial assignment
@whedon assign @username as editor

# Set the software archive DOI at the top of the issue e.g.
@whedon set 10.0000/zenodo.00000 as archive

# Set the software version at the top of the issue e.g.
@whedon set v1.0.1 as version

# Open the review issue
@whedon start review

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Remind an author or reviewer to return to a review after a
# certain period of time (supported units days and weeks)
@whedon remind @reviewer in 2 weeks

# Ask Whedon to do a  dry run of accepting the paper and depositing with Crossref
@whedon accept

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

EiC TASKS

# Invite an editor to edit a submission (sending them an email)
@whedon invite @editor as editor

# Reject a paper
@whedon reject

# Withdraw a paper
@whedon withdraw

# Ask Whedon to actually accept the paper and deposit with Crossref
@whedon accept deposit=true
matbesancon commented 4 years ago

@whedon check references

matbesancon commented 4 years ago

@whedon check references

matbesancon commented 4 years ago

So I can't get the DOI verification to work on this paper but it is all good on my side given the reviews and edits made by the authors. Thanks and sorry for the delay

AndreaNeumayr commented 4 years ago

@matbesancon I added all available DOI's for the reference list (ref.bib) in the paper. The JuliaCon proceedings template is not supporting it. There are no DOI's shown. Do you know how to fix that?

We discussed about the DOI's already.

Thank you, now worries.

AndreaNeumayr commented 4 years ago

@matbesancon the status is still under review. Do we still have to do something?

AndreaNeumayr commented 4 years ago

I read the whedon commands. I think the accept and accept deposit=true commands need to be executed. What do you think @matbesancon ?

matbesancon commented 4 years ago

thanks @AndreaNeumayr, the deposit requires an extra step on the side of the infrastructure that is not setup yet, but the paper is accepted as-is and will be archived soon, thanks a lot

vchuravy commented 4 years ago

@AndreaNeumayr thank you for your patience, is there an archival DOI for the code that you would like to add to the paper?

AndreaNeumayr commented 4 years ago

@vchuravy there's no archival DOI. But is it possible to tag it with the release number of Modia3D? Release v0.4.0 https://github.com/ModiaSim/Modia3D.jl/releases/tag/v0.4.0

ChrisRackauckas commented 4 years ago

Quickly setup a Zenodo?

matbesancon commented 4 years ago

Yes you will need to setup a zenodo and archive the current version, github itself is not an archiving process

AndreaNeumayr commented 4 years ago

Thanks, Modia3D v0.4.0 has a DOI now: 10.5281/zenodo.3978914

matbesancon commented 4 years ago

@whedon set 10.5281/zenodo.3978914 as archive

whedon commented 4 years ago

OK. 10.5281/zenodo.3978914 is the archive.

matbesancon commented 4 years ago

@whedon accept

whedon commented 4 years ago 
Attempting dry run of processing paper acceptance...
whedon commented 4 years ago

PDF failed to compile for issue #43 with the following error:

Can't find any papers to compile :-(

matbesancon commented 4 years ago

@whedon generate pdf from branch JuliaCon2019-paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch JuliaCon2019-paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

matbesancon commented 4 years ago

@whedon accept

whedon commented 4 years ago 
Attempting dry run of processing paper acceptance...
whedon commented 4 years ago

PDF failed to compile for issue #43 with the following error:

Can't find any papers to compile :-(

matbesancon commented 4 years ago

@whedon accept from branch JuliaCon2019-paper

whedon commented 4 years ago 
Attempting dry run of processing paper acceptance...