[PRE REVIEW]: Multiple Inference: A Python package for comparing multiple parameters

[editorialbot]

Submitting author: !--author-handle-->@dsbowen<!--end-author-handle-- (DILLON BOWEN) Repository: https://gitlab.com/dsbowen/conditional-inference Branch with paper.md (empty if default branch): Version: 1.0.0 Editor: !--editor-->@vissarion<!--end-editor-- Reviewers: @blakeaw, @mattpitkin, @nhejazi Managing EiC: Arfon Smith

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/7a2a4af277c0ad6ad6f41897f4489888"><img src="https://joss.theoj.org/papers/7a2a4af277c0ad6ad6f41897f4489888/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/7a2a4af277c0ad6ad6f41897f4489888/status.svg)](https://joss.theoj.org/papers/7a2a4af277c0ad6ad6f41897f4489888)

Author instructions

Thanks for submitting your paper to JOSS @dsbowen. Currently, there isn't an JOSS editor assigned to your paper.

The author's suggestion for the handling editor is @chartgerink.

@dsbowen if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

[editorialbot]

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.11 s (690.7 files/s, 104431.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          34            923           1257           2767
Jupyter Notebook                10              0           3219           1938
TeX                              1             27              0            242
Markdown                         6             60              0            139
YAML                             3             12             10            112
reStructuredText                13            256            130             83
make                             2             10             12             51
DOS Batch                        1              8              1             26
Bourne Shell                     3              7              6             20
INI                              1              2              0             15
TOML                             1              0              0              6
-------------------------------------------------------------------------------
SUM:                            75           1305           4635           5399
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 1196

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1257/pandp.20221065 is OK
- 10.1038/s41592-019-0686-2 is OK

MISSING DOIs

- 10.3386/w22193 may be a valid DOI for title: What motivates effort? Evidence and expert forecasts
- 10.3386/w12338 may be a valid DOI for title: Does price matter in charitable giving? Evidence from a large-scale natural field experiment
- 10.3386/w28726 may be a valid DOI for title: Selecting the most effective nudge: Evidence from a large-scale experiment on immunization
- 10.2139/ssrn.3689456 may be a valid DOI for title: An adaptive targeted field experiment: Job search assistance for refugees in Jordan
- 10.3386/w25147 may be a valid DOI for title: The opportunity atlas: Mapping the childhood roots of social mobility
- 10.3386/w23002 may be a valid DOI for title: The impacts of neighborhoods on intergenerational mobility II: County-level estimates
- 10.3386/w19843 may be a valid DOI for title: Where is the land of opportunity? The geography of intergenerational mobility in the United States
- 10.47004/wp.cem.2020.4320 may be a valid DOI for title: Inference on winners
- 10.2139/ssrn.563209 may be a valid DOI for title: Stepwise multiple testing as formalized data snooping
- 10.47004/wp.cem.2021.1721 may be a valid DOI for title: Inference for ranks with applications to mobility across neighborhoods and academic achievement across countries
- 10.1525/9780520313880-018 may be a valid DOI for title: Inadmissibility of the usual estimator for the mean of a multivariate normal distribution
- 10.1145/3292500.3330771 may be a valid DOI for title: Shrinkage estimators in online experiments
- 10.1214/08-aos630 may be a valid DOI for title: Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means
- 10.1596/27528 may be a valid DOI for title: Applying Behavioral Insights to Improve Tax Collection
- 10.25080/majora-92bf1922-011 may be a valid DOI for title: statsmodels: Econometric and statistical modeling with python
- 10.1111/rssb.12162 may be a valid DOI for title: The normal law under linear restrictions: simulation and estimation via minimax tilting
- 10.1109/wsc.2015.7408180 may be a valid DOI for title: Efficient probability estimation and simulation of the truncated multivariate student-t distribution

INVALID DOIs

- None

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[arfon]

:wave: @dsbowen, thanks for your submission to JOSS. As you noted in your submission form, there has been a related publication in the past. Could you share with me (e.g., a link to a diff on GitHub) the changes that have been made since the earlier publication?

[danielskatz]

@dsbowen - In addition, you could work on the possibly missing DOIs that editorialbot suggests, but note that some may be incorrect. Please feel free to make changes to your .bib file, then use the command @editorialbot check references to check again, and the command @editorialbot generate pdf when the references are right to make a new PDF. editorialbot commands need to be the first entry in a new comment.

[dsbowen]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.3386/w22193 is OK
- 10.3386/w12338 is OK
- 10.3386/w28726 is OK
- 10.2139/ssrn.3689456 is OK
- 10.3386/w25147 is OK
- 10.3386/w23002 is OK
- 10.3386/w19843 is OK
- 10.3386/w25456 is OK
- 10.1257/pandp.20221065 is OK
- 10.1111/j.1468-0262.2005.00615.x is OK
- 10.3386/w26883 is OK
- 10.1525/9780520313880-018 is OK
- 10.1145/3292500.3330771 is OK
- 10.1214/08-aos630 is OK
- 10.1596/27528 is OK
- 10.25080/majora-92bf1922-011 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.1111/rssb.12162 is OK
- 10.1109/wsc.2015.7408180 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[dsbowen]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[dsbowen]

@arfon and @danielskatz, Here is the last commit before we submitted the related publication: https://gitlab.com/dsbowen/conditional-inference/-/commit/35e2150b29d806389fc0c733b2dd70233ed404f2. I'm also attaching the git diff.

diff.txt

Thanks in advance for your patience if this isn't exactly what you were asking for - this is my first submission to JOSS.

[danielskatz]

I think https://gitlab.com/dsbowen/conditional-inference/-/compare/1.0.0...master?from_project_id=29049195 are the changes since the previously published version

[danielskatz]

However, the main code change I see is adding the _fit_bock function. I think my diff is missing a bunch of earlier commits, but I don't know gitlab well enough to figure this out - I want changes between 35e2150b and now, but I'm not getting all of them

[arfon]

For easier review of these changes, I pushed a copy of the code to GitHub (we can delete this as soon as we've assessed scope/size here): https://github.com/arfon/conditional-inference-joss/compare/35e2150b29d806389fc0c733b2dd70233ed404f2...main

[arfon]

@editorialbot query scope

@dsbowen – thanks for the extra context here. I would like to ask the broader editorial team for their input here. This will likely take a week or so. Thanks in advance for your patience!

[editorialbot]

Submission flagged for editorial review.

[danielskatz]

@dsbowen - is there an open version of the paper that editors can read? Maybe https://scholar.harvard.edu/files/iandrews/files/losers.pdf?

[dsbowen]

@danielskatz Yes, that's an open version of the paper that you can read. 35e2150b is the last commit before we submitted the Inference for Losers paper. There have been many changes besides _fit_bock since then!

I'd also like to note that an earlier version of some Bayesian estimators existed in 35e2150b, but those were not part of the Inference for Losers paper (as you'll see when you read it). Inference for Losers was only about the RankCondition estimator. Thanks for having the editorial team check it out.

[kyleniemeyer]

Hi @dsbowen, after further review the editorial board has decided this is good to proceed to review. We'll now find an editor and reviewers.

[kyleniemeyer]

@editorialbot invite @vissarion as editor

Hi @vissarion, are you available to edit this submission? Thanks!

[editorialbot]

Invitation to edit this submission sent!

[vissarion]

@editorialbot assign @vissarion as editor

[editorialbot]

Assigned! @vissarion is now the editor

[vissarion]

:wave: @adavidzh, @blakeaw, @Pqrs6, @mattpitkin, @nhejazi, @williamjameshandley would any of you be willing to review this submission for JOSS?

We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

[adavidzh]

👋 @adavidzh, @blakeaw, @Pqrs6, @mattpitkin, @nhejazi, @williamjameshandley would any of you be willing to review this submission for JOSS?

We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

I'm afraid I can't right now. Pinging @amarini in case he can get involved with JOSS.

[dsbowen]

Is it possible to amend the paper? I implemented a new method this week, and I'd like to add a sentence or two to the effect, "This package contains the first Python implementation of robust empirical Bayes confidence intervals (Armstrong et al., 2020)." No worries if it's too late in the review process.

[blakeaw]

Hi @vissarion, sure. I'm willing to review the submission.

[vissarion]

Is it possible to amend the paper? I implemented a new method this week, and I'd like to add a sentence or two to the effect, "This package contains the first Python implementation of robust empirical Bayes confidence intervals (Armstrong et al., 2020)." No worries if it's too late in the review process.

The review haven't started yet, so please go ahead and edit your paper!

[vissarion]

@editorialbot add @blakeaw as reviewer

[editorialbot]

@blakeaw added to the reviewers list!

[vissarion]

Hi @SuperKam91@matt-graham @johnveitch @dflemin3 @GregoryAshton @AvianaGlobal would any of you be willing to review this submission for JOSS?

We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

[mattpitkin]

@vissarion I'm able to review the paper

[vissarion]

@editorialbot add @mattpitkin as reviewer

[editorialbot]

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

[vissarion]

@editorialbot add @mattpitkin as reviewer

[editorialbot]

@mattpitkin added to the reviewers list!

[nhejazi]

@vissarion i’m able to review the package and paper too if that’s still helpful

[vissarion]

@editorialbot add @nhejazi as reviewer

[editorialbot]

@nhejazi added to the reviewers list!

[vissarion]

@editorialbot start review

[editorialbot]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/4492.

[vissarion]

Thanks @blakeaw, @mattpitkin, @nhejazi I have added you as reviewers. The review thread is https://github.com/openjournals/joss-reviews/issues/4492