[REVIEW]: NGSTrefftz: Add-on to NGSolve for Trefftz methods

whedon commented 2 years ago

Submitting author: !--author-handle-->@PaulSt@Nikoleta-v3<!--end-editor-- Reviewers: @mscroggs, @thelfer Archive: 10.5281/zenodo.6394628

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/c2f4e85b118c22b81aa27d7799265409"><img src="https://joss.theoj.org/papers/c2f4e85b118c22b81aa27d7799265409/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/c2f4e85b118c22b81aa27d7799265409/status.svg)](https://joss.theoj.org/papers/c2f4e85b118c22b81aa27d7799265409)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@mscroggs & @thelfer , please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @Nikoleta-v3 know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @mscroggs

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@PaulSt) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @thelfer

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

[ ] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[ ] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[ ] Repository: Is the source code for this software available at the repository url?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[ ] Contribution and authorship: Has the submitting author (@PaulSt) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[ ] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 2 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @mscroggs, @thelfer it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 2 years ago

Wordcount for paper.md is 546

whedon commented 2 years ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.05 s (843.4 files/s, 177599.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C++                             11            498            311           3260
C/C++ Header                     9            254             98           1158
Jupyter Notebook                11              0           1810            731
Python                           7            137            334            585
TeX                              1             14              0            119
YAML                             1              8              2            106
Markdown                         2             19              0             81
CMake                            1             10              5             54
Dockerfile                       2             15             15             40
JSON                             1              0              0             23
-------------------------------------------------------------------------------
SUM:                            46            955           2575           6157
-------------------------------------------------------------------------------

Statistical information for the repository '1d155acd4a973ae99ca7f710' was
gathered on 2022/02/07.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Christoph Lehrenfeld             1             2              2            0.00
Paul                           666         43191          36793           96.71
Paul Stocker                     5             9            442            0.55
PaulSt                          33          1392            871            2.74

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Christoph Lehrenfeld          2          100.0          0.0                0.00
Paul                       6633           15.4         15.1               10.58

whedon commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.camwa.2020.01.006 is OK
- 10.1007/s00211-017-0910-x is OK
- 10.1115/1.1995716 is OK
- 10.1016/0965-9978(95)00067-4 is OK
- 10.1007/978-3-319-41640-3_8 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

Nikoleta-v3 commented 2 years ago

👋🏼 @PaulSt @mscroggs & @thelfer this is the review thread for the paper. All of our communications will happen here from now on.

Both reviewers have checklists at the top of this thread with the JOSS requirements 🔝 As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention https://github.com/openjournals/joss-reviews/issues/4135 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 2-4 weeks. Please let me know if any of you require some more time. We can also use Whedon (our bot) to set automatic reminders if you know you'll be away for a known period of time.

Please feel free to ping me (@Nikoleta-v3 ) if you have any questions/concerns 👍🏻

whedon commented 2 years ago

:wave: @thelfer , please update us on how your review is going (this is an automated reminder).

whedon commented 2 years ago

:wave: @mscroggs, please update us on how your review is going (this is an automated reminder).

thelfer commented 2 years ago

I currently have a working version of NGSTrefftz, after resolution of various issues. Review starts now !

mscroggs commented 2 years ago

I've been busier than I expect the last two weeks. My review will be done next week

thelfer commented 2 years ago

@Nikoleta-v3 Would you remind me what I need to do to be able to check my checklist ? Sorry if I missed something ?

Nikoleta-v3 commented 2 years ago

👋🏻 @thelfer -- JOSS recently changed their editorial bot so please bear with me.

Could you try running/commenting the following command:

@editorialbot generate my checklist

It should generate a checklist for you as a comment on this issue, and you should be able to tick the checklist.

thelfer commented 2 years ago

Review checklist for @thelfer

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/PaulSt/NGSTrefftz?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@PaulSt) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

thelfer commented 2 years ago

@Nikoleta-v3. All my questions and comments have been answered by @PaulSt (See Issues 2 to 9). I consider my review as over and would recommend the publication of the paper.

I think that we can congratulate @PaulSt for his patience.

PaulSt commented 2 years ago

Thank you again @thelfer for the detailed review and very helpful feedback!

mscroggs commented 2 years ago

@PaulSt Nice software and nice paper!

I've opened a PR (https://github.com/PaulSt/NGSTrefftz/pull/10) to remove some brackets around references and add a reference to the paper that the FreeHyTe people wrote. Once that PR is resolve, I think this is ready to be accepted

Nikoleta-v3 commented 2 years ago

Thank you @thelfer and @mscroggs for your time and your reviews! @PaulSt once you have addressed the reviewers comments I am also going to have a final look 👍🏻

mscroggs commented 2 years ago

@Nikoleta-v3 My comments are all now dealt with and I've ticked all the boxes for my review

PaulSt commented 2 years ago

Thank you for your PR and the review @mscroggs!

Nikoleta-v3 commented 2 years ago

Again a huge thank you to the reviewers!

thelfer commented 2 years ago

Again a huge thank you to the reviewers!

My pleasure to contribute to JOSS.

Nikoleta-v3 commented 2 years ago

@whedon generate pdf

editorialbot commented 2 years ago

My name is now @editorialbot

Nikoleta-v3 commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 2 years ago

:warning: An error happened when generating the pdf.

Nikoleta-v3 commented 2 years ago

Hey @PaulSt 👋🏻 Could you please have a look at the following (minor) issues?

Documentation

On the NGSTrefftz website:

The hyperlinks to NGSolve do not work
The emojis :tada::+1: are not rendered

Readme

At the installation section you are listing the dependencies and their versions :+1: Could you include hyperlinks to the software’s website, or their installation pages?

This is more of a suggestion: why don’t you make Papers using the code section a section instead of a subsection?

Testing

I can see you have some tests (folder test) but I couldn't find any information on how to run the tests. Can you include this information in the documentation? And please point to it in, or just include it under Contributing to NGSTrefftz.

I can also see that you are running GitHub actions. Could you also run the test suite as part of the workflow?

PaulSt commented 2 years ago

Thank you @Nikoleta-v3 for pointing that out! I updated the website accordingly. The tests are run after building ngstrefftz on ubuntu during the "ubuntu" step of the github action.

Nikoleta-v3 commented 2 years ago

Thank you @PaulSt. Note that you left some commented out lines in the build.yml file.

At this point could you:

[ ] Let me know which tagged release of your software we should use (I am assuming v0.0.7).
[ ] Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository)
[ ] Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
[ ] Please list the DOI of the archived version here.

I can then move forward with accepting the submission. Let me know if you need any help with any of the above 😄

PaulSt commented 2 years ago

Hi @Nikoleta-v3, and sorry for the delay!

[X] I published a minor release which I would like to use for joss: v0.1.0
[X] The software is published in Zenodo at 10.5281/zenodo.6394628

Nikoleta-v3 commented 2 years ago

@editorialbot set 10.5281/zenodo.6394628 as archive

editorialbot commented 2 years ago

Done! Archive is now [10.5281/zenodo.6394628](https://zenodo.org/record/6394628)

Nikoleta-v3 commented 2 years ago

@editorialbot set v0.1.0 as version

editorialbot commented 2 years ago

Done! version is now v0.1.0

Nikoleta-v3 commented 2 years ago

@editorialbot recommend-accept

editorialbot commented 2 years ago

Attempting dry run of processing paper acceptance...

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.camwa.2020.01.006 is OK
- 10.1007/s00211-017-0910-x is OK
- 10.1115/1.1995716 is OK
- 10.1016/0965-9978(95)00067-4 is OK
- 10.1007/978-3-319-41640-3_8 is OK
- 10.1016/j.advengsoft.2018.03.014 is OK

MISSING DOIs

- None

INVALID DOIs

- None

editorialbot commented 2 years ago

:warning: Error prepararing paper acceptance.

PaulSt commented 2 years ago

Hi @Nikoleta-v3, I was just checking if the error was an issue in the paper but locally it compiled. Is it possible that

@editorialbot set 10.5281/zenodo.6394628 as archive

loops back onto itself? So one would need to call it without the link? :thinking:

Nikoleta-v3 commented 2 years ago

Thank you for checking the error! Let me try that 😄

Nikoleta-v3 commented 2 years ago

@editorialbot set 10.5281/zenodo.6394628 as archive

editorialbot commented 2 years ago

Done! Archive is now 10.5281/zenodo.6394628

Nikoleta-v3 commented 2 years ago

@editorialbot recommend-accept

editorialbot commented 2 years ago

Attempting dry run of processing paper acceptance...

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.camwa.2020.01.006 is OK
- 10.1007/s00211-017-0910-x is OK
- 10.1115/1.1995716 is OK
- 10.1016/0965-9978(95)00067-4 is OK
- 10.1007/978-3-319-41640-3_8 is OK
- 10.1016/j.advengsoft.2018.03.014 is OK

MISSING DOIs

- None

INVALID DOIs

- None

editorialbot commented 2 years ago

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/3099

If the paper PDF and the deposit XML files look good in https://github.com/openjournals/joss-papers/pull/3099, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

Nikoleta-v3 commented 2 years ago

Thank you @PaulSt for helping with the compiling issue 😄

PaulSt commented 2 years ago

Check final proof point_right openjournals/joss-papers#3099

@Nikoleta-v3 is missing as the editor in the proof, everything else looks good to me!

kyleniemeyer commented 2 years ago

@openjournals/dev I think the editor should be showing up properly in the proof, any idea what's going on there?

openjournals / joss-reviews