[REVIEW]: SigCorr: A Python package for studies of trials factors

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@vindex10@osorensen<!--end-editor-- Reviewers: @BSGalvan, @peifengjing, @gvieralopez Archive: 10.5281/zenodo.8096892

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/ba9741aeb587d6c3a9b49522a2dc8f8e"><img src="https://joss.theoj.org/papers/ba9741aeb587d6c3a9b49522a2dc8f8e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/ba9741aeb587d6c3a9b49522a2dc8f8e/status.svg)](https://joss.theoj.org/papers/ba9741aeb587d6c3a9b49522a2dc8f8e)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@BSGalvan & @peifengjing your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @osorensen know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @BSGalvan

📝 Checklist for @gvieralopez

📝 Checklist for @peifengjing

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1140/epjc/s10052-010-1470-8 is OK
- 10.1016/j.astropartphys.2011.08.005 is OK
- 10.48550/ARXIV.2206.12328 is OK
- 10.1140/epjc/s10052-011-1554-0 is OK

MISSING DOIs

- None

INVALID DOIs

- None

editorialbot commented 1 year ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.11 s (992.5 files/s, 139020.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          52           1048           1571           2065
Jupyter Notebook                16              0           7284           1495
reStructuredText                33            517            620            652
Markdown                         2             60              0            210
YAML                             1              7              3             56
TeX                              1              3              0             48
Bourne Shell                     5              7              1             27
make                             1              5              0              7
TOML                             1              0              0              2
-------------------------------------------------------------------------------
SUM:                           112           1647           9479           4562
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 1 year ago

Wordcount for paper.md is 1321

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

BSGalvan commented 1 year ago

Review checklist for @BSGalvan

Conflict of interest

[X] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[X] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[X] Repository: Is the source code for this software available at the https://gitlab.com/sigcorr/sigcorr?
[X] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[X] Contribution and authorship: Has the submitting author (@vindex10) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[X] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[X] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[X] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[X] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[X] Installation: Does installation proceed as outlined in the documentation?
[X] Functionality: Have the functional claims of the software been confirmed?
[X] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[X] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[X] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support
[X] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[X] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[X] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[X] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?

Software paper

[X] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[X] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[X] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[X] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[X] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

osorensen commented 1 year ago

👋 @jdalzatec, @amorenobr, could you please update us on how it's going with your reviews?

amorenobr commented 1 year ago

Review checklist for @amorenobr

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://gitlab.cern.ch/sigcorr/sigcorr?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@vindex10) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[ ] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[ ] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

osorensen commented 1 year ago

:wave: @jdalzatec could you please update us on how it's going with your review?

osorensen commented 1 year ago

@openjournals/dsais-eics, I just realized that the submitting authors and I are both at the University of Oslo. However, they are at the Department of Physics and I am at the Department of Psychology, we have never met nor collaborated on any papers, and we sit in different buildings. As far as I know our departments are not collaborating.

Is it ok if I continue as editor, or should we find another one?

gkthiruvathukal commented 1 year ago

@osorensen Thank you for the disclosure. Please continue as editor!

osorensen commented 1 year ago

👋 @jdalzatec could you please update us on how it's going with your review?

osorensen commented 1 year ago

@BSGalvan and @amorenobr, I notice you have some unchecked items in your review checklists. If these are issues the authors should fix, could you please elaborate here or open issues in the source repository? Thanks!

osorensen commented 1 year ago

👋 @jdalzatec could you please update us on how it's going with your review?

osorensen commented 1 year ago

@BSGalvan and @amorenobr, I notice you have some unchecked items in your review checklists. If these are issues the authors should fix, could you please elaborate here or open issues in the source repository? Thanks!

osorensen commented 1 year ago

👋 @jdalzatec could you please update us on how it's going with your review?

BSGalvan commented 1 year ago

Greetings, @osorensen! Terribly sorry for the delay in my review, the new year brought a load of unforeseen activities!

I am almost done with my checklist, but I have run into an obstacle I cannot seem to find the solution to. I wanted to raise issues related to certain items to add to the documentation (A statement of need, community guidelines, a list of dependencies & instructions regarding tests), but I am not able to do so. It seems the target repo's issue tracker requires a CERN Gitlab account, and seeing as I do not have one, I don't know how to proceed 😓

vindex10 commented 1 year ago

@BSGalvan hi! Thank you for your comment. I opened a ticket at CERN IT support regarding the access to the Issues, and I hope they will reply soon. The worst case we will need to move to GitHub :(

osorensen commented 1 year ago

Thanks for reporting this @BSGalvan, and thanks for working on fixing the issue, @vindex10.

You seem to be aware of this already @vindex10, but I'll anyway mention our community guidelines:

There should be clear guidelines for third-parties wishing to:

Contribute to the software

Report issues or problems with the software

Seek support

Looking forward to hearing back from you when the issue is fixed, @vindex10.

vindex10 commented 1 year ago

@editorialbot commands

editorialbot commented 1 year ago

Hello @vindex10, here are the things you can ask me to do:


# List all available commands
@editorialbot commands

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Check the references of the paper for missing DOIs
@editorialbot check references

# Perform checks on the repository
@editorialbot check repository

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for branch
@editorialbot set joss-paper as branch

# Generates the pdf paper
@editorialbot generate pdf

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Get a link to the complete list of reviewers
@editorialbot list reviewers

vindex10 commented 1 year ago

@osorensen Hi!

We received a reply from CERN IT support, CERN guest account doesn't give permission to open Issues.

I'm very sorry for the trouble, we didn't expect it to be a problem.

We moved our code with all the commits and workflows to gitlab: https://gitlab.com/sigcorr/sigcorr/-/tree/joss22 I hope the review process won't be "reset", because we preserved all the commit hashes. So nothing really has changed from the original setup, except for the links in the documentation (which now refer to gitlab.com instead of gitlab.cern.ch)

Would it be possible to change the "Repository" for this submission to the new one? We checked, Issues can be opened if one has a Gitlab account.

Thank you!

osorensen commented 1 year ago

Thanks for fixing this @vindex10. The review process will not be reset. I'll update the repository and it should be fine.

osorensen commented 1 year ago

@BSGalvan, feel free to open issues in the new source repository mentioned by @vindex10 above. I won't be able to update the repository listed in this review issue until I'm back in the office on Tuesday.

BSGalvan commented 1 year ago

@BSGalvan, feel free to open issues in the new source repository mentioned by @vindex10 above.

Here's the issue at the (new) target repo: [DOCS] Additional Sections for JOSS Review.

osorensen commented 1 year ago

Updated the respository information now. Testing if it works.

osorensen commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

vindex10 commented 1 year ago

@BSGalvan , thank you for your comments! we updated the docs accordingly.

I left some references in the comments under the issue: https://gitlab.com/sigcorr/sigcorr/-/issues/2

vindex10 commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

osorensen commented 1 year ago

👋 @amorenobr and @jdalzatec, can you please update us on how it's going with your reviews?

osorensen commented 1 year ago

👋 @amorenobr and @jdalzatec, can you please update us on how it's going with your reviews?

osorensen commented 1 year ago

@vindex10, FYI I have now contacted the reviewers by e-mail, so hopefully we will get some progress within the next few weeks.

osorensen commented 1 year ago

@vindex10, it turns out to be very hard to get in touch with the reviewers, so I'll have to try and find some new ones.

osorensen commented 1 year ago

👋 @jjerphan @yangli-stat @nickelnine37, would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

FYI, this review has already been going on for a while, but we try to find some new reviewers as we're not able to get in touch with the ones originally assigned.

jdalzatec commented 1 year ago

@editorialbot commands

editorialbot commented 1 year ago

Hello @jdalzatec, here are the things you can ask me to do:


# List all available commands
@editorialbot commands

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Check the references of the paper for missing DOIs
@editorialbot check references

# Perform checks on the repository
@editorialbot check repository

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for branch
@editorialbot set joss-paper as branch

# Generates the pdf paper
@editorialbot generate pdf

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Get a link to the complete list of reviewers
@editorialbot list reviewers

jdalzatec commented 1 year ago

@editorialbot list editors

editorialbot commented 1 year ago

List of JOSS editors

@AJQuinn
@AoifeHughes
@Bisaloo
@Fei-Tao
@Kevin-Mattheus-Moerman
@KristinaRiemer
@Nikoleta-v3
@adi3
@adonath
@ajstewartlang
@arfon
@atrisovic
@bmcfee
@britta-wstnr
@cMadan
@chartgerink
@christinahedges
@crsl4
@crvernon
@csoneson
@danasolav
@danielskatz
@dfm
@dhhagan
@diehlpk
@drvinceknight
@elbeejay
@eloisabentivegna
@emdupre
@fabian-s
@faroit
@fboehm
@fraukewiese
@galessiorob
@gkthiruvathukal
@graciellehigino
@hugoledoux
@ivastar
@jarvist
@jbytecode
@jedbrown
@jgostick
@jmschrei
@jsta
@kellyrowland
@kthyng
@kyleniemeyer
@lpantano
@lucydot
@luizirber
@majensen
@marcosvital
@martinfleis
@matthewfeickert
@mbobra
@melissawm
@mikldk
@mstimberg
@observingClouds
@olexandr-konovalov
@oliviaguest
@osorensen
@pdebuyl
@pibion
@plaplant
@ppxasjsm
@prashjha
@richardjgowers
@rkurchin
@samhforbes
@sbenthall
@spholmes
@timtroendle
@vissarion
@warrickball
@xuanxu
@zhubonan

jdalzatec commented 1 year ago

Review checklist for @jdalzatec

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[ ] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[ ] Repository: Is the source code for this software available at the https://gitlab.com/sigcorr/sigcorr?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[ ] Contribution and authorship: Has the submitting author (@vindex10) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[ ] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[ ] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[ ] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[ ] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

jdalzatec commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jjerphan commented 1 year ago

Hello @osorensen, thank you for the proposal.

Unfortunately, I am currently busy and do not have time for reviewing articles for JOSS at the moment.

osorensen commented 1 year ago

Ok, thanks for responding quickly @jjerphan

osorensen commented 1 year ago

:wave: @dflemin3, @1313e, @AvianaGlobal, would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

FYI, this review has already been going on for a while, but we try to find some new reviewers as we're not able to get in touch with the ones originally assigned.

osorensen commented 1 year ago

👋 @AoifeHughes, @pibion, @fdmatoz, would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

FYI, this review has already been going on for a while, but we try to find some new reviewers as we're not able to get in touch with the ones originally assigned.

vindex10 commented 1 year ago

btw, the theory paper (arxiv) related to this package has passed the peer review and is at the latest stages of the publishing process at JINST. This may be useful to get a larger scope and some real world applications of SigCorr.

What can be especially useful for this review process is the data published to zenodo: https://doi.org/10.5281/zenodo.7861345 you also can find a notebook there

this may save some time if one wants to try some functionality themself :) I hope this helps!

AoifeHughes commented 1 year ago

Apologies! My editor duties take up most of my time for JOSS and cannot review currently!

pibion commented 1 year ago

Apologies, I also am busy with JOSS editor duties and cannot edit. I would suggest contacting Erin Austen in the CU Denver math department - they specialize in statistics and may have faculty and/or graduate students who would be great for this topic. If they need any help with the code aspect, I'm local and would be happy to help out with any questions or issues.

openjournals / joss-reviews