Add subsection related to nonresponse, list more packages in weighting/calibration section

[bschneidr]

Hi Matthias,

This PR adds a small new subsection titled "response rates and nonresponse analysis", as a subsection of "Data Processing". The reason for implementing it as a subsection is because I didn't think it really fit into any of the other existing subsections: it's related to weighting and imputation, but is distinct enough that I didn't think it quite fit under either section. The reason for listing it under "Data Processing" is because it's typically the data producer that calculates the response rates and who conducts a nonresponse bias analysis.

A smaller, somewhat related change is that I listed a couple more packages under the weighting/calibration section. The addition of 'svrep' in particular is notable because it's the only package I'm aware of that implements sample-based calibration methods.

[matthias-da]

Sorry for the late response. Is now under discussion, but we need some time because of vacation from some of us.

[bschneidr]

No worries, there's nothing remotely urgent about this

[matthias-da]

Hi Ben

we all three think that the tools in PR are only of marginal interest and thus we tend to not add a subsection in the CTV on this.

However, we put svrep also (it was mentioned already in two other sections) to section Weighting.

Hope this is fine with you. Thanks again for your messages and proposals for extension

[bschneidr]

Hi @matthias-da,

Thanks for considering. I would disagree that tools for analyzing nonresponse bias are only of marginal interest for official statistics. For example, in the United States, federal statistical agencies that produce the nation's official statistics are generally required to report nonresponse rates and, when response rates are below some threshold (e.g., 80%) to conduct a nonresponse bias analysis. Of course, response rates are generally declining in the U.S. (as in other OECD countries), and so official statistical agencies are increasingly having to conduct nonresponse bias analyses. The 'nrba' package directly addresses this growing need among producers of official statistics. If we look at recent journal issues for official statistics (e.g., the Journal of Survey Statistics and Methodology) or look at agendas for recent JSM/ESRA conferences, we see lots of talks about NRBA. So this topic is clearly important to official statistics, and it's a good idea for the CTV to list one or more R packages directly focused on the topic of nonresponse and NRBA.

I think there's ambiguity about where in the CTV one would list 'nrba', but adding a subsection on nonresponse or at least just the 'nrba' package is an appropriate and valuable addition to this CTV.

Hope you're open to reconsidering, and thanks for your time.

-Ben

[matthias-da]

Hi Ben

sorry for the misunderstanding. I meant that the tools in RP are only of marginal interest from our point of view. I didn't mean that non-response analysis is not of interest - it has been of interest already for decades.

But one can do this straightforwardly in R with a few commands (sometimes one line of code) without the need for the package - doing so this might even be simpler when used to base R syntax, dplyr or data.table. At least that was my impression after looking at the package. If I am wrong, I am happy to see the benefits of comparing the package code with code without using the package. So, I wonder where is the true benefits of these packages?

We think that nrba, idcnrba, outcomerate are not super exciting now. Eg. outcomerate might just implement a standard of AAPOR (so very specific?), that might be relevant for a rather small community (?).

Maybe we have to reconsider a "misc" section to list packages that play an important role for a small community (AAPOR for example)?

If you think we are wrong, we are happy to reconsider our decision and we then would need to take a deep dive on it.

[bschneidr]

Thanks for being open to reconsidering. Below, I explain why the 'nrba' package is a good addition to this task view. 'idcnrba' and 'outcomerate' are more ambiguous, but the case for 'nrba' is clear-cut.

In any case, I wasn't aware that the standard for listing something on a Task View is whether it's "super exciting"; the overview page for CRAN task views says the following:

CRAN task views aim to provide guidance which packages on CRAN are relevant for tasks related to a certain topic. They give a brief overview of the included packages which can also be automatically installed using the ctv package. The views are intended to have a sharp focus so that it is sufficiently clear which packages should be included (or excluded) - and they are not meant to endorse the "best" packages for a given task. https://cran.r-project.org/web/views/

A CRAN package for nonresponse bias analysis for survey data certainly is relevant for official statistics. This package was developed by a large team of statisticians working in official statistics, and we use it in production for national and international official statistics programs (PIAAC, SDR, and others). And the 'nrba' package is the best (perhaps only?) package focused on nonreponse bias analysis, and thus is relevant to many many statisticians working in the production of official statistics.

So by the normal standards used for CRAN task views, it makes sense to include the 'nrba' package in the Official Statistics task view.

If you'd like more details about what the 'nrba' package offers, here's a list of things to consider:

• The package implements statistical methods which are useful for NRBA but complex. A few highlights:
  • Statistical tests comparing estimates from overlapping samples
  • Statistical tests comparing calibrated estimates to uncalibrated estimates
  • (To understand why these are deceptively complex statistical procedures, see this paper from the current issue of the Journal of Survey Statistics and Methodology)
  • Cumulative estimates (point estimates and design-based standard errors) for level-of-effort variables such as number of survey contact attempts
• For users who are new to NRBA methods, it provides a clear, well-documented suite of methods that are considered best practice. It contains several useful methodological notes and citations.
  • For example, see the documentation for the function t_test_of_weight_adjustment() and t_test_by_response_status()
  • Those are methods which are not straightforward to do in a few lines of base R or dplyr code. Moreover, because the methods are implemented as functions in a package, they are documented clearly and are thoroughly unit tested.
• It contains example datasets and code examples showing how to conduct an 'nrba'. So this has educational value and helps R users quickly implement an analysis.
• The functions work for a wide range of survey designs, including complex samples with stratification, clustering, etc. (which is something hard to analyze using just base R or dplyr).

I hope this helps explain why the 'nrba' package does belong on the Official Statistics task view.

[matthias-da]

Dear Ben

Of course we do not endorse the best packages only, but all relevant ones.

Thanks for giving more details on the package nrba. We definitely will have a closer look at it early next week and will come back to you on this matter. It is definitely worth a deeper dive into it.