search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
713 stars 38 forks source link

[REVIEW]: POSSA: Power simulation for sequential analyses and multiple hypotheses #4643

Closed editorialbot closed 2 years ago

editorialbot commented 2 years ago

Submitting author: !--author-handle-->@gasparl<!--end-author-handle-- (Gáspár Lukács) Repository: https://github.com/gasparl/possa/ Branch with paper.md (empty if default branch): Version: v0.6.1 Editor: !--editor-->@fboehm<!--end-editor-- Reviewers: @mingzehuang, @mmrabe Archive: 10.5281/zenodo.7027767

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/3018b5213b07c69049bec226949b29f7"><img src="https://joss.theoj.org/papers/3018b5213b07c69049bec226949b29f7/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/3018b5213b07c69049bec226949b29f7/status.svg)](https://joss.theoj.org/papers/3018b5213b07c69049bec226949b29f7)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@mingzehuang & @mmrabe, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @fboehm know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @mmrabe

📝 Checklist for @mingzehuang

editorialbot commented 2 years ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 2 years ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=0.06 s (444.5 files/s, 151670.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
HTML                             4            144              8           3424
R                               11            115            664           1929
Rmd                              4            356           1369            245
TeX                              1             11              0            105
Markdown                         3             31              0             61
YAML                             2             11              6             52
-------------------------------------------------------------------------------
SUM:                            25            668           2047           5816
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 2 years ago

Wordcount for paper.md is 339

editorialbot commented 2 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1002/ejsp.2023 is OK
- 10.1177/0146167220913363 is OK
- 10.5334/irsp.181 is OK
- 10.5964/meth.2811 is OK
- 10.1111/2041-210X.12504 is OK
- 10.1177/2515245920951503 is OK

MISSING DOIs

- 10.1016/c2013-0-10517-x may be a valid DOI for title: Statistical power analysis for the behavioral sciences

INVALID DOIs

- None
editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mmrabe commented 2 years ago

Review checklist for @mmrabe

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

mingzehuang commented 2 years ago

Review checklist for @mingzehuang

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

fboehm commented 2 years ago

@mmrabe and @mingzehuang - how are the reviews going? Please feel free to discuss here any revisions that the authors must make before publication. Thanks again!

mmrabe commented 2 years ago

Hi @fboehm ! I expect to finish the review by Tuesday. I hope that still works. Sorry for the delay.

mingzehuang commented 2 years ago

Hi, @gasparl, your package looks nice:) I see you have "contribution" section in your README file. I would suggest you also include code of conduct like many other open source packages:)

mingzehuang commented 2 years ago

Hi, @fboehm, I'm reviewing! Hopefully I can get it done by this weekend:)

mingzehuang commented 2 years ago

Hi, @gasparl, you have numerous great example with the link in your README:) Would you mind picking one simple example as an illustration of functionality and putting it in your JOSS paper like many other papers on JOSS?

fboehm commented 2 years ago

Thanks so much, @mmrabe and @mingzehuang ! Please feel free to ask me questions - here, in the comments - if you're unsure about anything. Thanks again!

gasparl commented 2 years ago

Sure.

gasparl commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mingzehuang commented 2 years ago

Hi, @gasparl, there is a requirement about automatic test in checklist. Would you mind using codecov and embedding the badge of coverage rate into your README.md?

gasparl commented 2 years ago

Hi, @gasparl, there is a requirement about automatic test in checklist. Would you mind using codecov and embedding the badge of coverage rate into your README.md?

Sure, I added it. (The relatively low percentage [52%] is because I included lots of warning messages for making the functions more foolproof, but these don't really need to be tested.)

gasparl commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mmrabe commented 2 years ago

The author @gasparl has written a very useful and flexible tool for simulation-based power analysis. The documentation and vignettes exceed the expected extent by far and provide many helpful examples. The submitted paper has become a lot stronger after adding a simple example, as @mingzehuang suggested.

Before I can check off all items on my checklist and recommend this software paper for publication, please attend to the following:

  1. I can not yet check off the license checkbox because the repo contains two license files. Please make sure they match. Issue: gasparl/possa#9
  2. In the example in the paper, I recommend that you add a citation for Pocock's correction. Issue: gasparl/possa#10

I would also like to make a minor suggestion, which may make the package more straightforward to use for the average R user and possibly increase the audience:

  1. Provide a wrapper function for common test objects that assists the user to extract the p-value. Issue: gasparl/possa#11
mmrabe commented 2 years ago

@gasparl already worked on the issues I posted to the repo as I was writing the lines above. So I can check off the remaining two boxes from my checklist right away. Thanks!

gasparl commented 2 years ago

Thank you very much @mmrabe for your kind and helpful review. Yes, I already answered the request (pardon my haste ;) ).

I'll also think about and address your suggestion, hopefully in the coming days.

mingzehuang commented 2 years ago

Thank you @gasparl ! I see the codecov!

gasparl commented 2 years ago

Thank you very much for your nice review, @mingzehuang!

gasparl commented 2 years ago

I now added the p value extraction function as suggested by @mmrabe.

@fboehm, this concludes my replies/revisions to all reviewer requests and comments.

mmrabe commented 2 years ago

I confirm that the new get_p(...) method is working and well documented. It has even found its way into the vignettes already. Thanks for adding this right away, @gasparl! I believe this will be very helpful to some potential users. I can without any concern recommend this paper for publication in JOSS. Great job!

gasparl commented 2 years ago

Thanks again @mmrabe !

fboehm commented 2 years ago

Thanks to the reviewers, @mmrabe and @mingzehuang for thorough and helpful reviews, and to @gasparl for timely implementation of the suggestions. The next step is for me to proofread the paper. I might have minor suggestions once I do that. I'll mention you, @gasparl, once I've completed this step.

fboehm commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fboehm commented 2 years ago

The paper looks really good. I have a minor correction that is needed:

line 50 in the pdf: capitalize R One way to do this is to put braces around capital R in your bib file: {R}

I have no other edits to suggest for the paper

fboehm commented 2 years ago

@editorialbot check references

editorialbot commented 2 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1002/ejsp.2023 is OK
- 10.1093/biomet/64.2.191 is OK
- 10.1177/0146167220913363 is OK
- 10.5334/irsp.181 is OK
- 10.5964/meth.2811 is OK
- 10.1111/2041-210X.12504 is OK
- 10.1177/2515245920951503 is OK

MISSING DOIs

- 10.1016/c2013-0-10517-x may be a valid DOI for title: Statistical power analysis for the behavioral sciences

INVALID DOIs

- None
fboehm commented 2 years ago

@gasparl - can you check to see if the suggested "missing DOI" is valid? If so, please add it to the bib file (and the resulting pdf). Thanks again!

fboehm commented 2 years ago

I manually checked the currently present DOIs, and they all resolved to the intended targets.

gasparl commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

gasparl commented 2 years ago
fboehm commented 2 years ago

Excellent, @gasparl! thanks for making those additions.

fboehm commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fboehm commented 2 years ago

The references all look good now. For the next steps, @gasparl, we need you to make a new release of the package and archive it, for example, with zenodo.org. Once you complete those tasks, please report here the version number and doi. Please ensure that the archive's author names and title match exactly those of the paper.pdf.

gasparl commented 2 years ago

Sure @fboehm, here it is: Version number: v0.6.1 DOI: https://doi.org/10.5281/zenodo.7027767

(The Zenodo badge with permanent DOI is also displayed on the GitHub repo.)

fboehm commented 2 years ago

@editorialbot set 10.5281/zenodo.7027767 as doi

editorialbot commented 2 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

fboehm commented 2 years ago

@editorialbot set 10.5281/zenodo.7027767 as archive

editorialbot commented 2 years ago

Done! Archive is now 10.5281/zenodo.7027767

fboehm commented 2 years ago

@editorialbot set v0.6.1 as version

editorialbot commented 2 years ago

Done! version is now v0.6.1