search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
703 stars 36 forks source link

[PRE REVIEW]: PathFinder: A Matlab/Octave package for oscillatory integration #6327

Closed editorialbot closed 2 months ago

editorialbot commented 7 months ago

Submitting author: !--author-handle-->@AndrewGibbs<!--end-author-handle-- (Andrew Gibbs) Repository: https://github.com/AndrewGibbs/PathFinder Branch with paper.md (empty if default branch): Version: v1.1.0 Editor: !--editor-->@olexandr-konovalov<!--end-editor-- Reviewers: @YehorYudinIPP, @fruzsinaagocs Managing EiC: Daniel S. Katz

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/851d659a3a85536bfc6b86de45a1641d"><img src="https://joss.theoj.org/papers/851d659a3a85536bfc6b86de45a1641d/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/851d659a3a85536bfc6b86de45a1641d/status.svg)](https://joss.theoj.org/papers/851d659a3a85536bfc6b86de45a1641d)

Author instructions

Thanks for submitting your paper to JOSS @AndrewGibbs. Currently, there isn't a JOSS editor assigned to your paper.

@AndrewGibbs if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands
editorialbot commented 7 months ago

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 7 months ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=0.12 s (780.2 files/s, 61707.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
MATLAB                          74            609           1037           2834
C/C++ Header                     3            152            416           1014
Markdown                         3            124              0            279
C                                4             79             86            278
TeX                              1             12              0            146
YAML                             4              1              4             46
Jupyter Notebook                 1              0              1              0
-------------------------------------------------------------------------------
SUM:                            90            977           1544           4597
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 7 months ago

Wordcount for paper.md is 1044

editorialbot commented 7 months ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.wavemoti.2023.103257 is OK
- 10.1016/j.jcp.2024.112787 is OK
- 10.1137/050636814 is OK
- 10.1137/18M1221138 is OK
- 10.1016/j.wavemoti.2018.09.015 is OK
- 10.1016/S0010-4655(00)00126-0 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 7 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 7 months ago

Five most similar historical JOSS papers:

(py)oscode: fast solutions of oscillatory ODEs Submitting author: @fruzsinaagocs Handling editor: @melissawm (Retired) Reviewers: @jakryd, @dlfivefifty Similarity score: 0.8092

OctApps: a library of Octave functions for continuous gravitational-wave data analysis Submitting author: @kwwette Handling editor: @danielskatz (Active) Reviewers: @stevenrbrandt Similarity score: 0.8081

BioSigKit: A Matlab Toolbox and Interface for Analysis of BioSignals Submitting author: @hooman650 Handling editor: @Kevin-Mattheus-Moerman (Active) Reviewers: @marianux, @fernandoandreotti, @bzenger Similarity score: 0.8018

ecopath_matlab: A Matlab-based implementation of the Ecopath food web algorithm Submitting author: @kakearney Handling editor: @Kevin-Mattheus-Moerman (Active) Reviewers: @roliveros-ramos Similarity score: 0.7990

RK-Opt: A package for the design of numerical ODE solvers Submitting author: @ketch Handling editor: @diehlpk (Active) Reviewers: @gardner48, @debdeepbh, @emconsta Similarity score: 0.7960

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

danielskatz commented 7 months ago

In the submission note from the author:

The algorithm behind the paper has recently been published in "Numerical evaluation of oscillatory integrals via automated steepest descent contour deformation", Journal of Computational Physics 501, 2024. That paper contained numerical experiments using an earlier version of PathFinder, and we gave roughly ten lines of sample code to highlight the versatility and robustness of the algorithm. However, that paper was primarily about the algorithm, not the software. When that paper was written, the code lacked detailed documentation, examples, unit testing, and a stable MEX compilation process. Moreover, it was not Octave compatible, so (in a sense) not open source. These issues have since been addressed, and I believe the software is now worthy of consideration for publication in JOSS.

This is https://doi.org/10.1016/j.jcp.2024.112787 with a preprint at https://arxiv.org/abs/2307.07261

danielskatz commented 7 months ago

👋 @AndrewGibbs - thanks for your submission. Before we proceed, I want the JOSS editorial board to weigh in on the previous publication. You should hear back in a week or two.

danielskatz commented 7 months ago

@editorialbot query scope

editorialbot commented 7 months ago

Submission flagged for editorial review.

danielskatz commented 6 months ago

👋 @AndrewGibbs - sorry for the delay in responding. The JOSS editors feel that this work is too similar to the previously published paper, and decline to review it.

danielskatz commented 6 months ago

@editorialbot reject

editorialbot commented 6 months ago

Paper rejected.

AndrewGibbs commented 6 months ago

Dear @danielskatz,

Thank you for your reply. I was surprised by this outright rejection, and I genuinely believe there has been a misunderstanding. I would be very grateful if the editors would take the time to read my short response below.

The grounds for outright rejection of the software by JOSS is due to its "similarity" to the paper describing the PathFinder algorithm that has been recently published by the Journal of Computational Physics (JCP). However, it is clear from the JCP scope that JCP is not a software journal and JCP does not review software or code. (This is in contrast to similarly named Elsevier journal - the Journal of Computational Science, which does review software.) Thus, while JCP reviewed the PathFinder algorithm and its mathematical justification, no version of the PathFinder software has previously been reviewed by JCP (or anyone).

Indeed, publication of the JCP paper was not contingent on the existence of a fully tested open-source software implementation - the JCP paper could have been published had it been supported by numerical results produced using a private, closed source, untested code that worked only on the examples considered in the JCP paper. I am asking JOSS, a software journal, to review my software implementation for publication. Obviously, the software is "similar" to the algorithm presented in the JCP article in the sense that it implements the algorithm, but this seems to me entirely consistent with the submission requirements for JOSS, which states:

Sometimes authors prepare a JOSS publication alongside a contribution describing a science application, details of algorithm development, and/or methods assessment. In this circumstance, JOSS considers submissions for which the implementation of the software itself reflects a substantial scientific effort. This may be represented by the design of the software, the implementation of the algorithms, creation of tutorials, or any other aspect of the software.

Surely, this situation must be extremely common for many JOSS submissions since JOSS papers are not long enough to allow a detailed description of the algorithm or any applications?

I believe that one of three things is happening:

  1. I am misunderstanding the JOSS scope.
  2. The ‘Co-publication of science, methods, and software’ subheading of the JOSS ‘Submission Requirements’ is inaccurate.
  3. The editors wrongly believe that the software has been reviewed previously.

I would be extremely grateful if the editors could provide some clarification as to which of these three is the case. And, if case 3, whether the paper could be reconsidered for review.

Thank you, Andrew

danielskatz commented 6 months ago

Thanks @AndrewGibbs - I'll bring this back to the editors and ask them to take another look at it.

I also apologize for my overly brief message. I was just returning from vacation and trying to deal with too many things at once. I should have also said

Our decision does not mean that your software is not useful to the research community. Please see https://joss.readthedocs.io/en/latest/submitting.html#other-venues-for-reviewing-and-publishing-software-packages for other suggestions for how you might receive credit for your work.

AndrewGibbs commented 6 months ago

Thank you @danielskatz, this is much appreciated.

danielskatz commented 6 months ago

@AndrewGibbs - We have reconsidered, and in particular, discussed the issue of duplicate submission in some detail. In this case, since your paper was an algorithm paper and not a software paper, which was not full appreciated by all of us initially, we have decided to accept this as in scope for review, and I will proceed with this process as soon as possible.

AndrewGibbs commented 6 months ago

@danielskatz thank you very much for reconsidering.

danielskatz commented 6 months ago

👋 @olexandr-konovalov - would you be able to edit this submission?

danielskatz commented 6 months ago

@editorialbot invite @olexandr-konovalov as editor

editorialbot commented 6 months ago

Invitation to edit this submission sent!

olexandr-konovalov commented 6 months ago

@editorialbot generate pdf

editorialbot commented 6 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

olexandr-konovalov commented 6 months ago

@editorialbot assign @olexandr-konovalov as editor

editorialbot commented 6 months ago

Assigned! @olexandr-konovalov is now the editor

danielskatz commented 4 months ago

👋 @olexandr-konovalov - what's going on with this review? Are you looking for reviewers off-line?

olexandr-konovalov commented 3 months ago

Hello @kwwette and @@ketch - would one or both of you be able to review this submission for JOSS, please?

If not, could you possibly recommend someone whom we may approach regarding this?

ketch commented 3 months ago

I'm sorry but I'm truly overwhelmed with other review requests right now. I don't know someone who works in this particular area.

olexandr-konovalov commented 3 months ago

Hi @YehorYudinIPP - would you be able to review this submission for JOSS, please? If not, could you possibly recommend someone whom we may approach regarding this?

YehorYudinIPP commented 3 months ago

Hi @YehorYudinIPP - would you be able to review this submission for JOSS, please? If not, could you possibly recommend someone whom we may approach regarding this?

I have some time on my hands for that, but I have never reviewed a software paper before.

olexandr-konovalov commented 3 months ago

@YehorYudinIPP thanks for the reply - the review takes place in a GitHub issue, is open, and author-reviewer-editor conversations are encouraged. JOSS reviews involve downloading and installing the software, and inspecting the repository and submitted paper for key elements (see https://joss.readthedocs.io/en/latest/review_criteria.html for further details). Editors and reviewers post comments on the Review issue, and authors respond to the comments and improve their submission until acceptance (or withdrawal, if they feel unable to satisfy the review). You can find more information in the reviewer's guide at https://joss.readthedocs.io/en/latest/, and also see some other review issues in this repository for actual examples of JOSS reviews (in particular, reviewer's checklists). If that looks good and you agree to review for JOSS, that will be much appreciated. Then the review will start when we will have two reviewers.

kwwette commented 3 months ago

Hello @kwwette and @@ketch - would one or both of you be able to review this submission for JOSS, please?

If not, could you possibly recommend someone whom we may approach regarding this?

Unfortunately I'm very busy for the next month or so. If you're still looking for a 2nd reviewer in July, ping me again and I might be able to help.

olexandr-konovalov commented 3 months ago

Hello @zzyatlantise - I have found your details in JOSS reviewers database. Would you be able to review this submission for JOSS, please?

YehorYudinIPP commented 3 months ago

@YehorYudinIPP thanks for the reply - the review takes place in a GitHub issue, is open, and author-reviewer-editor conversations are encouraged. JOSS reviews involve downloading and installing the software, and inspecting the repository and submitted paper for key elements (see https://joss.readthedocs.io/en/latest/review_criteria.html for further details). Editors and reviewers post comments on the Review issue, and authors respond to the comments and improve their submission until acceptance (or withdrawal, if they feel unable to satisfy the review). You can find more information in the reviewer's guide at https://joss.readthedocs.io/en/latest/, and also see some other review issues in this repository for actual examples of JOSS reviews (in particular, reviewer's checklists). If that looks good and you agree to review for JOSS, that will be much appreciated. Then the review will start when we will have two reviewers.

I took a look at the criteria, and it should be doable, not until the end of the month, though. It still feels a bit strange that the reviewers are discussed in a GitHub chat open to the author.

olexandr-konovalov commented 3 months ago

@YehorYudinIPP I see - I've found your details in our reviewers database, but it was not clear to me whether do you have any previous experience of reviewing for JOSS. Yes, that's a unique model (and I think open review perfectly aligns with the meaning of "O" in "JOSS"). It was described, for example, in https://doi.org/10.7717/peerj-cs.147 and https://doi.org/10.1109/MCSE.2018.03221930.

The review starts when I will find a 2nd reviewer, so it might not happen this month anyway.

YehorYudinIPP commented 3 months ago

@YehorYudinIPP I see - I've found your details in our reviewers database, but it was not clear to me whether do you have any previous experience of reviewing for JOSS. Yes, that's a unique model (and I think open review perfectly aligns with the meaning of "O" in "JOSS"). It was described, for example, in https://doi.org/10.7717/peerj-cs.147 and https://doi.org/10.1109/MCSE.2018.03221930.

The review starts when I will find a 2nd reviewer, so it might not happen this month anyway.

Thank you for the links! Indeed I have no experience with JOSS reviewing so far. I will look forward to the second reviewer!

olexandr-konovalov commented 3 months ago

Hello @fruzsinaagocs - we are looking for a reviewer for this paper. Would you be able to review this submission for JOSS, please? If not, could you possibly recommend someone whom we may approach regarding this?

fruzsinaagocs commented 2 months ago

Hi All! Yes, I'd be happy to review this submission.

olexandr-konovalov commented 2 months ago

Hi @fruzsinaagocs - many thanks! I will start the review now.

olexandr-konovalov commented 2 months ago

@editorialbot add @YehorYudinIPP as reviewer

editorialbot commented 2 months ago

@YehorYudinIPP added to the reviewers list!

olexandr-konovalov commented 2 months ago

@editorialbot add @fruzsinaagocs as reviewer

editorialbot commented 2 months ago

@fruzsinaagocs added to the reviewers list!

olexandr-konovalov commented 2 months ago

@editorialbot start review

editorialbot commented 2 months ago

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/6902.

olexandr-konovalov commented 2 months ago

Many thanks again @YehorYudinIPP and @fruzsinaagocs for agreeing to review this paper! Further conversation moves to #6902 - please see it for further instructions, generating your review checklists etc.