[REVIEW]: BCEA: An R Package for Cost-Effectiveness Analysis

editorialbot commented 2 years ago

Submitting author: !--author-handle-->@n8thangreen@ajstewartlang<!--end-editor-- Reviewers: @rowlandseymour, @MikeLydeamore Archive: 10.5281/zenodo.7040450

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/c98500946568218ac5b70bba60c88c2b"><img src="https://joss.theoj.org/papers/c98500946568218ac5b70bba60c88c2b/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/c98500946568218ac5b70bba60c88c2b/status.svg)](https://joss.theoj.org/papers/c98500946568218ac5b70bba60c88c2b)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@rowlandseymour & @volkerschmid, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @ajstewartlang know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @rowlandseymour

📝 Checklist for @MikeLydeamore

editorialbot commented 2 years ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 2 years ago

Checking the BibTeX entries failed with the following error:

No paper file path

editorialbot commented 2 years ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.72 s (474.7 files/s, 117145.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                      22           6903           6843          24358
HTML                           149           4866           1077          21276
R                              121           1792           4428           6955
CSS                             10            208            136            940
Markdown                         6            203              0            699
Rmd                             20            534            908            532
XML                              1              0              0            450
TeX                              2             62              8            303
YAML                             5             43             70            169
Sass                             1              4              0             71
SVG                              2              0              1             65
JSON                             1              0              0              1
-------------------------------------------------------------------------------
SUM:                           340          14615          13471          55819
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 2 years ago

Failed to discover a Statement of need section in paper

editorialbot commented 2 years ago

:warning: An error happened when generating the pdf.

ajstewartlang commented 2 years ago

@whedon generate pdf from branch JOSS-paper

editorialbot commented 2 years ago

My name is now @editorialbot

ajstewartlang commented 2 years ago

@editorialbot generate pdf from branch JOSS-paper

editorialbot commented 2 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

ajstewartlang commented 2 years ago

@editorialbot commands

editorialbot commented 2 years ago

Hello @ajstewartlang, here are the things you can ask me to do:


# List all available commands
@editorialbot commands

# Add to this issue's reviewers list
@editorialbot add @username as reviewer

# Remove from this issue's reviewers list
@editorialbot remove @username from reviewers

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Assign a user as the editor of this submission
@editorialbot assign @username as editor

# Remove the editor assigned to this submission
@editorialbot remove editor

# Remind an author or reviewer to return to a review after a 
# certain period of time (supported units days and weeks)
@editorialbot remind @reviewer in 2 weeks

# Check the references of the paper for missing DOIs
@editorialbot check references

# Perform checks on the repository
@editorialbot check repository

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for version
@editorialbot set v1.0.0 as version

# Set a value for archive
@editorialbot set 10.21105/zenodo.12345 as archive

# Set a value for branch
@editorialbot set joss-paper as branch

# Generates the pdf paper
@editorialbot generate pdf

# Recommends the submission for acceptance
@editorialbot recommend-accept

# Flag submission with questionable scope
@editorialbot query scope

# Get a link to the complete list of reviewers
@editorialbot list reviewers

# Open the review issue
@editorialbot start review

ajstewartlang commented 2 years ago

@editorialbot set joss-paper as branch

editorialbot commented 2 years ago

Done! branch is now joss-paper

ajstewartlang commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:warning: An error happened when generating the pdf.

ajstewartlang commented 2 years ago

@editorialbot set branch to JOSS-paper

editorialbot commented 2 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

ajstewartlang commented 2 years ago

@editorialbot set JOSS-paper as branch

editorialbot commented 2 years ago

Done! branch is now JOSS-paper

ajstewartlang commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

rowlandseymour commented 2 years ago

Review checklist for @rowlandseymour

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/n8thangreen/BCEA/?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@n8thangreen) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

rowlandseymour commented 2 years ago

I've completed my review of the package and overall I think this is a very well executed package and submission for JOSS. The package is already on CRAN and has a user base meaning that you meet a high standard for usability and contribution. There is a book on using the package, which is the ultimate form of documentation. Running goodpractice and dev tools checks highlight only very minor concerns, showing how well the package is written. I was able to run the functions and generate output and plots.

I have raised five issues, none of which are major, two are minor and the rest I would consider at the authors' discretion. The first minor issue is about guiding the user towards vignettes as I don't think it is obvious where they should begin and some of the vignettes are very sparse. The second issue concerns contributor guidelines and I don't think they meet JOSS' current standards.

Thanks to @n8thangreen and the other authors, you made this very straightforward to review.

n8thangreen commented 2 years ago

I believe I have now addressed all of the review comments. Thanks @rowlandseymour for the time and effort to review. I have closed the issues in the BCEA repo corresponding to the JOSS review.

What are the next steps, please? Is there something else that I need to now do? Thanks!

ajstewartlang commented 2 years ago

Thanks @n8thangreen and many thanks @rowlandseymour for your review - we'll wait now for comments from @volkerschmid who will be able to take a look at this submission in a week or two.

rowlandseymour commented 2 years ago

Thanks @n8thangreen, I think the table of contents in the vignette is an excellent edition.

@ajstewartlang All the items on my checklist are now ticked off.

ajstewartlang commented 2 years ago

:wave: @volkerschmid Just checking in to see that you're getting on ok with your review of this submission?

volkerschmid commented 2 years ago

Review checklist for @volkerschmid

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[ ] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[ ] Repository: Is the source code for this software available at the https://github.com/n8thangreen/BCEA/?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[ ] Contribution and authorship: Has the submitting author (@n8thangreen) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[ ] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

volkerschmid commented 2 years ago

👋 @volkerschmid Just checking in to see that you're getting on ok with your review of this submission?

Sorry the review got delayed due to Covid infections in my family. I'll get to it ASAP.

ajstewartlang commented 2 years ago

:wave: @volkerschmid would it be possible for you to complete your review within the next 2 weeks please? I'd like to have this submission wrapped up by the end of June. Thanks.

n8thangreen commented 2 years ago

Hi @ajstewartlang, @volkerschmid Has there been any progress with the review, please? Is there anything you'd like me to do? Thanks! N

ajstewartlang commented 2 years ago

@volkerschmid will you be able to complete your review of this submission please? If you can't, please let me know and I'll try and find an alternative reviewer.

ajstewartlang commented 2 years ago

@editorialbot add @MikeLydeamore as reviewer

editorialbot commented 2 years ago

@MikeLydeamore added to the reviewers list!

ajstewartlang commented 2 years ago

@editorialbot remove @volkerschmid as reviewer

editorialbot commented 2 years ago

@volkerschmid removed from the reviewers list!

MikeLydeamore commented 2 years ago

Review checklist for @MikeLydeamore

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/n8thangreen/BCEA/?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@n8thangreen) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

MikeLydeamore commented 2 years ago

I have completed my review of the package. Overall I agree with the previous reviewer. The package is very thoroughly documented, works as expected and has good development practices.

I believe it to be a valuable addition to the journal.

My only recommendation is the naming of a few of the included datasets be renamed to avoid overriding R functions with datasets.

ajstewartlang commented 2 years ago

Many thanks for your review and for your helpful comments @MikeLydeamore - I very much appreciate it! @n8thangreen could you address the datasets naming issue and let me know when that's done please?

n8thangreen commented 2 years ago

I've changed the e, c and pi objects in the package supplied data to eff, cost and pi_post. I've pushed to the default branch dev. Is this sufficient? Thanks.

ajstewartlang commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ajstewartlang commented 2 years ago

@editorialbot check references

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s40273-019-00837-x is OK
- 10.1177/0272989X18754513 is OK
- 10.1101/670612 is OK
- 10.1016/S0167-6296(98)00039-3 is OK
- 10.1007/s40273-018-0697-3 is OK
- 10.1177/0272989X12458348 is OK

MISSING DOIs

- 10.1016/j.jval.2014.08.1988 may be a valid DOI for title: Bayesian models for cost-effectiveness analysis in the presence of structural zero costs
- 10.3386/t0227 may be a valid DOI for title: Net health benefits: a new framework for the analysis of uncertainty in cost effectiveness analysis
- 10.1002/hec.617 may be a valid DOI for title: A framework for cost-effectiveness analysis from clinical trial data
- 10.1002/sim.861 may be a valid DOI for title: Bayesian cost effectiveness analysis from clinical trial data
- 10.1214/10-sts351 may be a valid DOI for title: A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data
- 10.1016/b978-0-08-051581-6.50057-x may be a valid DOI for title: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images
- 10.1214/ss/1177011136 may be a valid DOI for title: Inference from iterative simulation using multiple sequences
- 10.1111/j.1467-9868.2010.00765.x may be a valid DOI for title: Riemann manifold Langevin and Hamiltonian Monte Carlo methods

INVALID DOIs

- None

ajstewartlang commented 2 years ago

@editorialbot generate pdf

ajstewartlang commented 2 years ago

@editorialbot check references

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ajstewartlang commented 2 years ago

@editorialbot check references

ajstewartlang commented 2 years ago

@openjournals/dev any idea why the check references command isn't working on this submission? The pdf builds ok and has the reference list in it. The .bib file looks ok to me.

ajstewartlang commented 2 years ago

@editorialbot check references

openjournals / joss-reviews