Closed whedon closed 2 years ago
Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @bonh, @WilkAndy it looks like you're currently assigned to review this paper :tada:.
:warning: JOSS reduced service mode :warning:
Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.
:star: Important :star:
If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿
To fix this do the following two things:
For a list of things I can do to help you, just type:
@whedon commands
For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:
@whedon generate pdf
Software report (experimental):
github.com/AlDanial/cloc v 1.88 T=0.29 s (370.5 files/s, 58814.9 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Python 66 2737 5300 6037
reStructuredText 39 860 743 1407
Markdown 2 24 0 19
make 1 4 6 9
-------------------------------------------------------------------------------
SUM: 108 3625 6049 7472
-------------------------------------------------------------------------------
Statistical information for the repository '15e27ad81d067e4382f138f9' was
gathered on 2021/09/21.
The following historical commit information, by author, was found:
Author Commits Insertions Deletions % of changes
Alex Vasile 1 1992 1031 12.42
elizabethmonte 6 17212 4099 87.58
Below are the number of rows from each author that have survived and are still
intact in the current revision:
Author Rows Stability Age % in comments
Alex Vasile 1992 100.0 0.0 24.60
elizabethmonte 12082 70.2 1.4 25.00
PDF failed to compile for issue #3742 with the following error:
Can't find any papers to compile :-(
@whedon generate pdf from branch publications
Attempting PDF compilation from custom branch publications. Reticulating splines etc...
:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:
@nasserma , @elizabethmonte and @Alex-Vasile - i have a question about authorship. You can read the JOSS guidelines here: https://joss.readthedocs.io/en/latest/submitting.html#authorship . Given those guidelines, are you OK with James and Nasser being co-authors, even though they appear to have contributed zero code?
@nasserma , since i am a heavy contributor to MOOSE i don't like your statement "Open-source packages ... such as MOOSE ... have very steep learning curves and are inaccessible to the general scientific and engineering communities." MOOSE was designed to be easy to use. Perhaps it has changed into something more complicated over time. Anyway, your statement is just your opinion. Feel free to make no changes, but I guess i'd tone it down a little, by perhaps changing to "... can have steep learning curves." (without the "inaccessible" part).
@nasserma , i have not yet read your paper in detail, nor downloaded the software, played with it, etc, but overall i think this paper should be significantly longer. You may have satisfied the minimum requirements for a JOSS paper, but i feel to excite readers to try your software, you need more details. I think you should mention:
This is really summarizing work you've already done, so i hope it won't be a huge chore for you.
@nasserma , i'm going on holiday for the next ~10 days, so won't do any more on this for a while, but hopefully you can address the above concerns while i'm away.
Oh, just one final point: your title "multiphysics package" gives the wrong idea to me. When i think of "multiphysics", i'm thinking of solid mechanics + fluid mechanics (of all kinds) + heat transfer + electromagnetics, etc. But your code, i think is just Navier Stokes. Can you comment here, or perhaps change the title?
:wave: @WilkAndy, please update us on how your review is going (this is an automated reminder).
:wave: @bonh, please update us on how your review is going (this is an automated reminder).
@lucydot @WilkAndy This is our first time submitting to JOSS, I just want to confirm how we respond to reviewer comments. Should we be collating them and responding after they are complete or should we respond to each one individually? We have a response ready for @WilkAndy but it also involves revisions to the manuscript.
@nasserma I am interested in your response to the points raised by @WilkAndy too.
@lucydot I am assessing the scholarly effort right now and will require some time on that point alone.
Hi @nasserma - The JOSS review process is a little different as it is more discussion based. It is very common for some issues/points to be raised, which are then responded to by the authors, then another issue/point is raised, with another response and so-on. Your response does not need to be as formal as for other more traditional journals. So please feel free to respond now to the issues raised already, and make any adjustments needed to the manuscript - this can be updated as needed as we progress with the review.
Hello @bonh I can see you raised an issue in the JOSS reviews repo. Raising issues is a good way to progress the review (we want to avoid long discussions on this thread) but please create a new issue in the target repository (https://github.com/uw-comphys/opencmp) and link to those issues (especially acceptance-blockers) by leaving comments here. Thanks - Lucy.
Just trying to install and use. Please add instructions for getting "NGSolve version 6.2.2101" to your "IO" docs
Just trying to install and use. Please add instructions for getting "NGSolve version 6.2.2101" to your "IO" docs
I will update the website over the weekend, for now please refer to NGSolves download page,
for installation instructions. The newest version available 6.2.2105 should work, but the Windows, MacOS, and Ubuntu PPA all provide access to installers for older versions, including 6.2.2101.
Hi @nasserma - The JOSS review process is a little different as it is more discussion based. It is very common for some issues/points to be raised, which are then responded to by the authors, then another issue/point is raised, with another response and so-on. Your response does not need to be as formal as for other more traditional journals. So please feel free to respond now to the issues raised already, and make any adjustments needed to the manuscript - this can be updated as needed as we progress with the review.
Thanks for the clarification, I'll post responses to all of the comments (except for those explicitly referring to other authors) today.
@nasserma , since i am a heavy contributor to MOOSE i don't like your statement "Open-source packages ... such as MOOSE ... have very steep learning curves and are inaccessible to the general scientific and engineering communities." MOOSE was designed to be easy to use. Perhaps it has changed into something more complicated over time. Anyway, your statement is just your opinion. Feel free to make no changes, but I guess i'd tone it down a little, by perhaps changing to "... can have steep learning curves." (without the "inaccessible" part).
We have relaxed the language in the revised manuscript, as suggested. Our intention was never to diminish other software packages' utility, but rather differentiate OpenCMP's purpose. Our core mandate is to enable users who are familiar with the command-line interface to perform simulations without having to do any programming whatsoever, while simultaneously empowering developers to contribute new multiphysics models.
@nasserma , i have not yet read your paper in detail, nor downloaded the software, played with it, etc, but overall i think this paper should be significantly longer. You may have satisfied the minimum requirements for a JOSS paper, but i feel to excite readers to try your software, you need more details. I think you should mention:
* code coverage * perhaps outline the tutorials * a bit more details about the user interface * a picture of an example, perhaps explaining why your code/approach is advantageous in terms of usability, accuracy, computational efficiency, etc, compared with other codes. Sometimes it is hard to not denigrate excellent alternative codes when you're doing this, but you needn't go over-the-top: you're just trying to show the reader your code is nice.
This is really summarizing work you've already done, so i hope it won't be a huge chore for you.
Based on the JOSS requirements for submissions, "the paper should be between 250-1000 words" and the submitted manuscript is at the upper limit. We could add more details into the paper, but it would be at the expense of other text, most of which has been added to meet the requirements of what the paper should include from the JOSS website,
https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements
We have added references to the OpenCMP website in the revised manuscript. The website content addresses all of your comments except for code coverage, which we have added to the revised manuscript without exceeding the 1000 word limit.
Oh, just one final point: your title "multiphysics package" gives the wrong idea to me. When i think of "multiphysics", i'm thinking of solid mechanics + fluid mechanics (of all kinds) + heat transfer + electromagnetics, etc. But your code, i think is just Navier Stokes. Can you comment here, or perhaps change the title?
The term "multiphysics" has a relatively broad definition of which yours is definitely included. From our perspective, coupling fluid flow, heat transfer, mass transfer, and chemical reaction is inclusive of the term "multiphysics". Please see Tutorial 9 from the website for an example of this. Additionally, we are in the process of development (dispersed) multiphase multicomponent flow models which will further support the use of the classification of the package as supporting "multiphysics".
We do not plan on contributing solid mechanics models but hope that future contributions will now that we have developed a website and a publicly accessible software repository.
@nasserma , @elizabethmonte and @Alex-Vasile - i have a question about authorship. You can read the JOSS guidelines here: https://joss.readthedocs.io/en/latest/submitting.html#authorship . Given those guidelines, are you OK with James and Nasser being co-authors, even though they appear to have contributed zero code?
First, a note about code contributions to the project. The project began on a private internal repo, with commits being periodically being rolled up and copied into the GitHub repo. This results in actual code contributions being unintentionally obscured on the GitHub repo. Future commits will better reflect authorship as development branches are switched from internal repos to GitHub.
To address your question directly, yes I am okay with James and Nasser being co-authors given that they have been active participants. James has contributed code to the repository in the past and is actively implementing new models at the moment. Nasser, as well as James, has provided direction in terms of code structure, and help in understanding, implementing, debugging the numerical methods.
@nasserma , @elizabethmonte and @Alex-Vasile - i have a question about authorship. You can read the JOSS guidelines here: https://joss.readthedocs.io/en/latest/submitting.html#authorship . Given those guidelines, are you OK with James and Nasser being co-authors, even though they appear to have contributed zero code?
First, a note about code contributions to the project. The project began on a private internal repo, with commits being periodically being rolled up and copied into the GitHub repo. This results in actual code contributions being unintentionally obscured on the GitHub repo. Future commits will better reflect authorship as development branches are switched from internal repos to GitHub.
To address your question directly, yes I am okay with James and Nasser being co-authors given that they have been active participants. James has contributed code to the repository in the past and is actively implementing new models at the moment. Nasser, as well as James, has provided direction in terms of code structure, and help in understanding, implementing, debugging the numerical methods.
I agree with @Alex-Vasile, the paper’s current set of authors accurately reflects contributions to OpenCMP.
*edit to clarify meaning
Hi @nasserma!
I am assessing the scholar effort of your code. In my opinion the performance of simulation codes is a very important feature (the most sophisticated implementation of a wonderful physical model will not have a wider impact if the solution time is unacceptable). About the performance of your code (haven't tested it yet): Is it possible to run your code in parallel? In the paper you mention, that utilizing MPI is planned but are you able to use shared memory? Have you compared the performance of your code against other codes? Do you have performed three-dimensional simulations?
On the diffuse interface method:
In the paper you reference this arxiv article https://arxiv.org/abs/2101.06824 for the diffuse interface method. This article shares a lot of the authors but the software OpenCMP is not mentioned. Did you use a different implementation there? If not, I think it would be a good idea to cross-reference your papers. In the present joss paper a clear reference to a more physics and simulation results focused paper would help the interested reader to find details of your work.
Secondly, using the diffuse interface method to represent immersed boundaries is not a new idea presented for the first time in arxiv:2101.06824, right? Perhaps you should add a reference to the paper which indicates this and use your reference to refer the reader to the improvements you made to the method.
You mention that there are no other implementations of the diffuse interface method. Is that really correct? How about the Phase Field implementation of comsol? Can that not be used to simulate "hard" boundaries by choosing super high values for density and viscosity or so?
@nasserma , i have not yet read your paper in detail, nor downloaded the software, played with it, etc, but overall i think this paper should be significantly longer. You may have satisfied the minimum requirements for a JOSS paper, but i feel to excite readers to try your software, you need more details. I think you should mention:
* code coverage * perhaps outline the tutorials * a bit more details about the user interface * a picture of an example, perhaps explaining why your code/approach is advantageous in terms of usability, accuracy, computational efficiency, etc, compared with other codes. Sometimes it is hard to not denigrate excellent alternative codes when you're doing this, but you needn't go over-the-top: you're just trying to show the reader your code is nice.
This is really summarizing work you've already done, so i hope it won't be a huge chore for you.
Based on the JOSS requirements for submissions, "the paper should be between 250-1000 words" and the submitted manuscript is at the upper limit. We could add more details into the paper, but it would be at the expense of other text, most of which has been added to meet the requirements of what the paper should include from the JOSS website,
https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements
We have added references to the OpenCMP website in the revised manuscript. The website content addresses all of your comments except for code coverage, which we have added to the revised manuscript without exceeding the 1000 word limit.
Please don't worry overly about the 1000 word limit. I think it's much more important that this paper accurately reflect the awesomeness of your code, otherwise all your work might fail to attract interest. For what it's worth, my most recent contribution to JOSS is 3280 words long. Please rest assured that unless we find something obviously wrong (like you've simply copied your code from somewhere else!) this paper will almost certainly be published, but we're just trying to make it better.
Hi @nasserma as @WilkAndy mentioned the 1000 word limit is not strict (however I wouldn't advise going as high as 3000 words plus 😉). I'd also like to re-iterate the idea mentioned by Andy that this review process is collaborative and is focused on improving your code and documentation before publication.
This discussion thread is getting quite long so I'd encourage @WilkAndy @bonh to raise any other issues on the repo issues tracker (https://github.com/uw-comphys/opencmp/issues). Personally I find it easier to track what has been raised and whether it has been resolved this way.
I can see @bonh raised a few questions 2 weeks ago, just in case you missed them @nasserma.
I can see @bonh raised a few questions 2 weeks ago, just in case you missed them @nasserma.
@lucydot Please apologize the delay, the other authors and I having been dealing with teaching, conferences, etc and plan on completing our response/revision in the next few weeks.
I can see @bonh raised a few questions 2 weeks ago, just in case you missed them @nasserma.
@lucydot Please apologize the delay, the other authors and I having been dealing with teaching, conferences, etc and plan on completing our response/revision in the next few weeks.
You're going to conferences?! What sort of lucky world do you live in!
@nasserma do you have a timeline for responding to the reviewer comments? I can see that issues have been raised for each one, though no actions yet taken. If you are unsure how best to proceed, please feel free to discuss and ask questions here.
@nasserma do you have a timeline for responding to the reviewer comments? I can see that issues have been raised for each one, though no actions yet taken. If you are unsure how best to proceed, please feel free to discuss and ask questions here.
@lucydot I finish with lectures on Dec 7th and will have responses and a revised manuscript ready for December 10th. Most of this is ready but waiting on some finishing touches, but this semester has been much more difficult than usual because we are teaching and TAing with both in-person and online sections for each course.
Comment: define your physics in Table(1) through equations . Eg "multi-component reacting NS" might mean lots of things.
Just trying to install on a fresh windows machine. Also need:
pip install scipy
I think automated tests are mandatory, and i can't find any. You have tutorial examples - great, a new user can test things are working to some degree, but it wouldn't take much work to build a test suite out of those, so that new users can be confident things are working as planned. For instance in "tutorial 4", various norms are outputted, but these are different for my computer than are shown at https://opencmp.io/tutorials/tutorial_4.html . While i'm not too concerned at the differences, it would be much easier and confidence-inspiring if new users could just write
python run_all_tests.py
and OpenCMP replies back: "all tests passed". I use the python unittest
.
Various tutorials fail to run. These could be because i'm running on a vanilla windows system with no extra packages installed.
Refs https://github.com/uw-comphys/opencmp/issues/12 Refs https://github.com/uw-comphys/opencmp/issues/13 Refs https://github.com/uw-comphys/opencmp/issues/14 Refs https://github.com/uw-comphys/opencmp/issues/15
I haven't read the documentation carefully, I'm sorry. However, at the moment i believe all your tutorials are "simple" cases that can be solved easily using other software. If true, this doesn't really compel me to try your new software. I believe the big new thing about your software is diffuse boundary method. How about highlighting that through an awesome example, both in your online documentation and this JOSS paper.
I think automated tests are mandatory, and i can't find any. You have tutorial examples - great, a new user can test things are working to some degree, but it wouldn't take much work to build a test suite out of those, so that new users can be confident things are working as planned. For instance in "tutorial 4", various norms are outputted, but these are different for my computer than are shown at https://opencmp.io/tutorials/tutorial_4.html . While i'm not too concerned at the differences, it would be much easier and confidence-inspiring if new users could just write
python run_all_tests.py
and OpenCMP replies back: "all tests passed". I use the python
unittest
.
Please see the README.md file which describes our unit test procedure. We use pytest instead of unittest for unit and integration tests. We prefer it for a few reasons, but if you see the pytests/ directory there is also a README.md file with further information regarding how (i) how run the existing tests and (ii) how to create new tests.
At this point we have quite a decent amount of tests, they are all present in subdirectories of pytests/ and are commented/documented.
That being said, the most recent push to the Github branch seems to have caused some significant issues with pytests and the tutorials. We are in the process of working out how to deal with our internal VCS and external VCS (Github). I will work this out with the others ASAP.
Various tutorials fail to run. These could be because i'm running on a vanilla windows system with no extra packages installed.
Refs uw-comphys/opencmp#12 Refs uw-comphys/opencmp#13 Refs uw-comphys/opencmp#14 Refs uw-comphys/opencmp#15
Please use revision ID a475159 for now, there was definitely an issue pushing changes from the last revision and we will resolve in shortly.
I will continue my review after we have discussed uw-comphys/opencmp#6 and uw-comphys/opencmp#7. I rate these as critical to assess the scholarly effort.
I will continue my review after we have discussed uw-comphys/opencmp#6 and uw-comphys/opencmp#7. I rate these as critical to assess the scholarly effort.
We are just working out a few last issues incurred from migrating to setuptools to address the installation issue brought up by @WilkAndy and we will push a version that should address all comments, except for the distributed memory parallelization one.
If I understand correctly, @bonh wants us to have MPI supported for the paper/software to be further considered. Hopefully we can get feedback on our changes in response to all of the other comments and get a few weeks to work on MPI.
I am sorry for not being more clear: recommending the paper for publication in JOSS and improving the package are two different parts of my work here, I think. So, as I wrote just now in uw-comphys/opencmp#6,
in my opinion, a CFD package which is not able to at least simulate moderate scale problems AND does not implement some new methodology does not meet the "scholarly effort" described by JOSS.
I just want to engage in a discussion with you about your work! Now if you are able to simulate moderate size problems (e.g., something in 3D or so, using shared memory) this would be okay for me to proceed with the review. Same way if you convince me of the novelty of your method (diffuse interface). However, I think that if you offer the user MPI (or some other distributed memory parallelization), this would greatly improve your software and therefore the paper! But it is not mandatory for proceeding with the review (in contrast, the ability to solve moderate problems, shared or distributed, is mandatory for me, see above, if the method is not significantly new).
I really hope this clarifies my approach here...
@bonh @WilkAndy @lucydot Please see the recent revision that we pushed, it addresses many of the comments that were made, although there are a few outstanding issues that I would like to discuss before we address them.
Please see the open issues,
https://github.com/uw-comphys/opencmp/issues
and our responses. Once we have your feedback we'll make the next round of revisions and please note that the installation instructions have changed,
@whedon generate pdf
PDF failed to compile for issue #3742 with the following error:
[WARNING] Could not convert image '/tmp/tex2pdf.-9d76ffa7cd6dcdaa/6bfdb6359d3c5367d353cc654cde8d34a5a41722.svg': check that rsvg-convert is in path.
rsvg-convert: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)
Error producing PDF.
! LaTeX Error: Cannot determine size of graphic in /tmp/tex2pdf.-9d76ffa7cd6dcd
aa/6bfdb6359d3c5367d353cc654cde8d34a5a41722.svg (no BoundingBox).
See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...
l.627 ...b6359d3c5367d353cc654cde8d34a5a41722.svg}
Looks like we failed to compile the PDF
@whedon generate pdf from branch publications
Attempting PDF compilation from custom branch publications. Reticulating splines etc...
PDF failed to compile for issue #3742 with the following error:
[WARNING] Could not convert image '/tmp/tex2pdf.-949c2ce0db16a535/6bfdb6359d3c5367d353cc654cde8d34a5a41722.svg': check that rsvg-convert is in path.
rsvg-convert: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)
Error producing PDF.
! LaTeX Error: Cannot determine size of graphic in /tmp/tex2pdf.-949c2ce0db16a5
35/6bfdb6359d3c5367d353cc654cde8d34a5a41722.svg (no BoundingBox).
See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...
l.627 ...b6359d3c5367d353cc654cde8d34a5a41722.svg}
Looks like we failed to compile the PDF
@lucydot @nasserma , am i doing something wrong? Please help me to generate the pdf.
@bonh I will fix this shortly, sorry for the oversight.
Submitting author: !--author-handle-->@nasserma<!--end-author-handle-- (Nasser Mohieddin Abukhdeir) Repository: https://github.com/uw-comphys/opencmp Branch with paper.md (empty if default branch): publications Version: v1.0.0 Editor: !--editor-->@lucydot<!--end-editor-- Reviewers: @bonh, @WilkAndy Archive: 10.5281/zenodo.6515912
:warning: JOSS reduced service mode :warning:
Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@bonh & @WilkAndy, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @lucydot know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Review checklist for @bonh
✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨
Conflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
Review checklist for @WilkAndy
✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨
Conflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper