Improve manuscript for JOSS Review

TomKellyGenetics commented 3 years ago

Review: manuscript

Overall

Firstly, let me thank the authors for their efforts to prepare an excellent package. It's certainly of wide interest for the research community and particularly providing a Shiny App is a great way to make it accessible to those without programming expertise. As R is popular in statistics, life sciences and biomedical research, an R package would be widely used in these fields.

I agree that PR #64 significantly improves the manuscript as discussion in Issue #61. Overall the paper nicely summarises the functionality of the package and provides some examples of use cases. I have some minor feedback on structure and grammar. Generally, I believe the paper is worth of acceptance with minor editing.

Contributions

[x] Contribution and authorship: Has the submitting author (@aghaynes) made major contributions to the software Does the full list of paper authors seem appropriate and complete?

[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

As shown here @aghaynes has prepared the manuscript and got feedback other authors. Authorship seems to be appropriate to me.

Structure

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?

[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?

These are both well covered but I'd recommend merging them into one "Summary" section to avoid repetition. For example lines 9, 19, and 22 are similar.

Introduction

[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?

Normally I'd expect some discussion of alternatives but none seem to be available for "precision" sample sizes (or confidence intervals). I'd be willing to make an exception to this as under the circumstances but it could be helpful to provide some examples of "power" calculations for hypothesis testing. JOSS manuscripts are meant to be short but I think some comparisons here would be informative and related software (e.g., for power calculations in R) should be cited. As pointed out the paper this is complementary work.

Usage

As discussed in #61 I think this section is appropriate, especially as long-form vignette documentation (discussed here: #66) is not yet available.

Grammar

The writing is overall of high quality and easy to understand for non-experts. Some statistical jargon is unavoidable in this kind of work. There are a few sections that could be rephrased for clarity. These should be minor changes that will not take long. Please read the manuscript carefully for similar issues.

line 9, 20: we have therefore...

line 13: Sample size is a crucial step in for planning a clinical study.

line 13-14: A ~~too small~~ study that is too small leads to inconclusive results.

line 18: man research projects aim at to...

line 22: We ~~programmed~~ have developed an R package that...

Change "first" and "second" to "firstly" and "secondly".

line 35: we have also

line 42: can be installed and loaded into the R session...

line 53: we have developed

~~line 61: We often observe in our consulting activity that researchers try to implement a hypothesis-based approach~~

line 64 fascilitate -> facilitate, adequate -> appropriate

[x] correct spelling and grammar (above)

References

References are formatted correctly but a few notable citations are missing.

[x] Cite R packages for power calculations
[x] Cite the R core team

> citation()

To cite R in publications use:

  R Core Team (2020). R: A language and environment for statistical computing. R Foundation for
  Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

A BibTeX entry for LaTeX users is

  @Manual{,
    title = {R: A Language and Environment for Statistical Computing},
    author = {{R Core Team}},
    organization = {R Foundation for Statistical Computing},
    address = {Vienna, Austria},
    year = {2020},
    url = {https://www.R-project.org/},
  }

We have invested a lot of time and effort in creating R, please cite it when using it for data
analysis. See also ‘citation("pkgname")’ for citing R packages.

Questions

A limitation of the package is that it does not allow calculating the probability of a CI, i.e. the probability that a future confidence interval will have at least the desired precision.

It's great that the authors have acknowledged clearly and as it stands the packages is already very useful. Are there plans to add this in the future? How difficult would it be? Maintaining an R package is a commitment and I understand that time and funding to support this may be limited. As mentioned in the paper, investing in this will reduce the burden on consulting activities and aid researchers that do not have this expertise available.

This completes my review of the manuscript. I will test the latest version of the software and shiny app to complete the JOSS review.

aghaynes commented 3 years ago

Thanks for the additional comments @TomKellyGenetics. I have incorporated them into #73.

I merged the statement of need and summary and remodelled it slightly to improve the flow. I fixed the other comments as you indicated with the exception of

line 61: We often observe in our consulting activity that researchers try to implement a hypothesis-based approach

as I don't see anything wrong there (MS Word also sees no issue with it...).

I highlight the pwr and TrialSize packages and also the Clinical Trials task view which lists a large number of packages that might be of relevance (although none cover precision based methods to our knowledge). I also cite R - thanks for pointing that out!

Furthermore, I also add you, @amoeba and @majensen to the acknowlegdements.

Regarding your question about the probability of an estimate being within the CI... there is a paper describing the method in the appendix if I remember but we need to think hard about how to do it for each of the different methods. It is something that we are considering adding in the future though, when time/resources allow. Stata actually has the option for the few methods available there, although their code is proprietory of course and not (easily) accessible, if at all.

TomKellyGenetics commented 3 years ago

I've checked the updated paper and everything seems to be taken care of. It's a great concise summary of the package.

As a stylistic suggestion, I think this section would fit better in the Introductory section (following paragraph). It's a bit detailed for the "Summary".

There are many software packages for hypothesis-based sample size calculation, such as Stata, PASS, G*Power, including many R packages, such as pwr (@pwr) andTrialSize (@trialsize; see other packages detailed on the CRAN Clinical Trials taskview). To the best of our knowledge, only Stata provides precision-based approaches, and only then for a small number of statistics.

I accept the manuscript as is and leave this change to the author's discretion.

CTU-Bern / presize