[REVIEW]: adaptr: an R package for simulating and comparing adaptive clinical trials

editorialbot commented 2 years ago

Submitting author: !--author-handle-->@agranholm@fabian-s<!--end-editor-- Reviewers: @elimillera, @stmcg Archive: 10.5281/zenodo.6490087

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/e070045b6998360d93eed06c95ba44a7"><img src="https://joss.theoj.org/papers/e070045b6998360d93eed06c95ba44a7/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/e070045b6998360d93eed06c95ba44a7/status.svg)](https://joss.theoj.org/papers/e070045b6998360d93eed06c95ba44a7)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@elimillera & @stmcg, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @fabian-s know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @stmcg

📝 Checklist for @elimillera

editorialbot commented 2 years ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 2 years ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.07 s (781.3 files/s, 138974.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Markdown                        10            371              0           2559
R                               27            391           1873           2325
SVG                              5              0              0            452
Rmd                              4            212            571            229
TeX                              1             11              0            110
YAML                             5             25             10            110
-------------------------------------------------------------------------------
SUM:                            52           1010           2454           5785
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 2 years ago

Wordcount for paper.md is 2765

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1186/s12916-018-1017-7 is OK
- 10.1007/s00134-021-06587-9 is OK
- 10.1186/s13063-021-05130-x is OK
- 10.1177/1740774519877836 is OK
- 10.1002/pst.2015 is OK
- 10.21105/joss.01686 is OK
- 10.1038/s41573-019-0034-3 is OK
- 10.1016/j.jclinepi.2020.04.025 is OK
- 10.1186/s13063-019-4021-0 is OK
- 10.1136/bmjopen-2018-024256 is OK

MISSING DOIs

- None

INVALID DOIs

- None

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fabian-s commented 2 years ago

👋🏼 @agranholm @elimillera @stmcg this is the review thread for the paper. All of our communications will happen here from now on.

@elimillera @stmcg please begin by creating checklists for your reviews by entering

@editorialbot generate my checklist

as a comment in this thread.

These checklists contain the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines in the top comment in this thread.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#REVIEW_NUMBER so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 2-4 weeks. Please let me know if any of you require some more time. We can also use EditorialBot (our bot) to set automatic reminders if you know you'll be away for a known period of time.

Ping me (@fabian-s) if you have any questions/concerns.

stmcg commented 2 years ago

Review checklist for @stmcg

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/INCEPTdk/adaptr?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@agranholm) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

elimillera commented 2 years ago

Review checklist for @elimillera

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/INCEPTdk/adaptr?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@agranholm) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

stmcg commented 2 years ago

This is an excellent package. One comment on the checklist item: "State of the field: Do the authors describe how this software compares to other commonly-used packages?"

The authors mention that existing software for simulating adaptive clinical trial designs is either closed source, has limited features, or is based on graphical user interfaces. It would be helpful if the authors state the key additional features of the adaptr package compared to existing open source software.

agranholm commented 2 years ago

Thank you very much @stmcg !

@fabian-s what is the process from here, should we wait for the second review to complete before revising the manuscript file? Thanks!

fabian-s commented 2 years ago

@fabian-s what is the process from here, should we wait for the second review to complete before revising the manuscript file? Thanks!

you can, but you can also go ahead and address this now. since we coordinate the review process in this thread and have an open line of communication, we don't need to rely on "review rounds" and can tackle issues as they come up.

agranholm commented 2 years ago

@editorialbot generate pdf

We have now updated the manuscript to respond to the comment by @stmcg . We have expanded the third paragraph of the discussion (first part of the third paragraph), to add a (brief) comparison of adaptr with other open-source alternatives, with reference to a recent systematic review summarising the features of the available software solutions.

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fabian-s commented 2 years ago

@stmcg

We have expanded the third paragraph of the discussion (first part of the third paragraph), to add a (brief) comparison of adaptr with other open-source alternatives, with reference to a recent systematic review summarising the features of the available software solutions.

Diff: https://github.com/INCEPTdk/adaptr/commit/acc8df586054a57620d0f7d5d8f6b137bf892344

does the edit address your concerns? if so, please check that item off.

fabian-s commented 2 years ago

:wave: @elimillera, please let us know your timeframe for this review

elimillera commented 2 years ago

Hey @fabian-s , My workload is looking better. I'm hoping to get this review done this weekend

elimillera commented 2 years ago

I second @stmcg. A very well developed package. @fabian-s My review is done and everything looks good.

fabian-s commented 2 years ago

@editorialbot generate pdf

fabian-s commented 2 years ago

@editorialbot check references

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1186/s12916-018-1017-7 is OK
- 10.1007/s00134-021-06587-9 is OK
- 10.1186/s13063-021-05130-x is OK
- 10.1177/1740774519877836 is OK
- 10.1002/pst.2015 is OK
- 10.21105/joss.01686 is OK
- 10.1038/s41573-019-0034-3 is OK
- 10.1016/j.jclinepi.2020.04.025 is OK
- 10.1186/s13063-019-4021-0 is OK
- 10.1136/bmjopen-2018-024256 is OK

MISSING DOIs

- None

INVALID DOIs

- None

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fabian-s commented 2 years ago

@agranholm, seems like we're almost good to go!

At this point could you:

[x] Make a tagged release of your software, and list the version tag of the archived version here.
[x] Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository)
[x] Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
[x] Please list the DOI of the archived version here.

I can then move forward with accepting the submission.

agranholm commented 2 years ago

@elimillera Thank you very much!

@fabian-s Excellent.

We have previously made a tagged release on GitHub of the first (and newest version), v1.0.0: https://github.com/INCEPTdk/adaptr/releases/tag/v1.0.0

This version of the software - v1.0.0 - has now been archived at Zenodo and metadata should be correct and match the submission. The DOI is: 10.5281/zenodo.6490087, full link: https://doi.org/10.5281/zenodo.6490087

Thank you very much for moving this forward so quickly!

fabian-s commented 2 years ago

@editorialbot set 10.5281/zenodo.6490087

fabian-s commented 2 years ago

@editorialbot set 10.5281/zenodo.6490087 as archive @editorialbot set v1.0.0 as version

fabian-s commented 2 years ago

@editorialbot recommend-accept

note for JOSS EiCs: this paper exceeds the recommended size for JOSS papers somewhat (~2000 words), but IMO this is adequate here: there's lots of code examples in the text and the paper does not contain any "fluff" that would be easy to remove without losing important content.

danielskatz commented 2 years ago

👋 @fabian-s - trying again - @editorialbot was on a break

danielskatz commented 2 years ago

@editorialbot recommend-accept

note for JOSS EiCs: this paper exceeds the recommended size for JOSS papers somewhat (~2000 words), but IMO this is adequate here: there's lots of code examples in the text and the paper does not contain any "fluff" that would be easy to remove without losing important content.

editorialbot commented 2 years ago

Paper is not ready for acceptance yet, the archive is missing

danielskatz commented 2 years ago

@editorialbot set 10.5281/zenodo.6490087 as archive

editorialbot commented 2 years ago

Done! Archive is now 10.5281/zenodo.6490087

danielskatz commented 2 years ago

@editorialbot set v1.0.0 as version

editorialbot commented 2 years ago

Done! version is now v1.0.0

danielskatz commented 2 years ago

@fabian-s - also note that there can only be one @editorialbot command per comment, and it has to be the first thing in the comment

danielskatz commented 2 years ago

@editorialbot recommend-accept

note for JOSS EiCs: this paper exceeds the recommended size for JOSS papers somewhat (~2000 words), but IMO this is adequate here: there's lots of code examples in the text and the paper does not contain any "fluff" that would be easy to remove without losing important content.

editorialbot commented 2 years ago

Attempting dry run of processing paper acceptance...

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1186/s12916-018-1017-7 is OK
- 10.1007/s00134-021-06587-9 is OK
- 10.1186/s13063-021-05130-x is OK
- 10.1177/1740774519877836 is OK
- 10.1002/pst.2015 is OK
- 10.21105/joss.01686 is OK
- 10.1038/s41573-019-0034-3 is OK
- 10.1016/j.jclinepi.2020.04.025 is OK
- 10.1186/s13063-019-4021-0 is OK
- 10.1136/bmjopen-2018-024256 is OK

MISSING DOIs

- None

INVALID DOIs

- None

editorialbot commented 2 years ago

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/3178

If the paper PDF and the deposit XML files look good in https://github.com/openjournals/joss-papers/pull/3178, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

danielskatz commented 2 years ago

@fabian-s - please check this and let me know when you think it's ready for me to proofread and then potentially publish

fabian-s commented 2 years ago

@danielskatz thanks for your help.

I'm sorry to say I missed one tiny thing during proof-reading earlier -- it's

"Planning adaptive trials and comparing adaptive trials designs is complex, however, and require statistical" instead of "Planning adaptive trials and comparing adaptive trials designs is complex, however, and requires statistical", see https://github.com/INCEPTdk/adaptr/pull/3.

Not sure if we can't proceed anyway?

danielskatz commented 2 years ago

👋 @agranholm - can you make this ☝️ change, and I'll also proofread and see if I find anything else?

agranholm commented 2 years ago

Thanks, I’ve merged @fabian-s ‘ pull request, so the typo is corrected now, thanks for catching this. Not sure if I should regenerate the PDF now at this stage using the usual command or if it should be done by one of you?

epiben commented 2 years ago

Just chipping in; hope that's fine. The sentence has a grammatical error, but it's not that one. It should read, "Planning adaptive trials and comparing adaptive trials designs are complex, however, and require statistical". Planning + comparing => third person plural. Thus, "require" is correct.

danielskatz commented 2 years ago

@agranholm - you can regenerate the PDF if you want, if not I will

danielskatz commented 2 years ago

@epiben - thanks for your suggestion - Either way would be fine (alt: Planning adaptive trials and comparing adaptive trials designs is complex, however, and requires statistical) but the singular/plural part should be consistent. @agranholm - can you also change this, and I will still proofread this whole thing later today?

agranholm commented 2 years ago

@editorialbot generate pdf

Thanks both. @danielskatz I have updated it according to @epiben s suggestion (plural in both places) and regenerated the PDF.

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

danielskatz commented 2 years ago

@editorialbot accept

editorialbot commented 2 years ago

Doing it live! Attempting automated processing of paper acceptance...

editorialbot commented 2 years ago

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

openjournals / joss-reviews