vincentarelbundock
commented
2 years ago Thanks for the details @go-bayes!
One reason I'm skeptical that we don't need to account for between-imputation variances in general, is that in the multiple imputation vignette we need to fill-in values for the regressors as well, and not just for potential outcomes. So there are two different sources of uncertainty that we need to account for.
In any case, bootstrapping the estimates the extremely easy, as I now illustrate in this vignette: https://vincentarelbundock.github.io/marginaleffects/articles/sandwich.html#bootstrap
Since this is a software development website, and not a general statistics discussion forum, I don't expect we'll resolve all of those issues here. However, if you feel that some of the results produced by marginaleffects are formally incorrect, or if you have specific feature requests, I'll be happy to look into them.
Thanks a lot for engaging with the package! I really appreciate it!
go-bayes
Originally posted by @go-bayes in https://github.com/vincentarelbundock/marginaleffects/issues/49#issuecomment-1209142313
Thanks Vincent and contributors to this thread. As food for thought, consider Westreich and colleagues 2015 paper: https://academic.oup.com/ije/article/44/5/1731/2594566?login=false
Ignore that the authors compare a very specific multiple imputation approach for counterfactual recovery with a standard, no frills, G-computation approach. Nevertheless, the authors make the following comments that would seem to apply quite generally to causal inference over multiply imputed data.
"... note that we are only estimating a mean, so accounting for between-imputation variance is unnecessary; and related ... Rubin’s formula for the variance cannot be used because every observation contributes to both exposed and unexposed calculations, and therefore we bootstrap. A closed-form variance estimator is likely possible though not explored here..."
Following Westreich et al's logic, it is not entirely clear to me whether and how Rubin's rules apply when we are estimating counterfactual contrasts using G-computation. I am curious to follow this issue; it will certainly interest many others. Thanks again all.
Thanks for replying Vincent. I share your intuitions! However, when it comes to causal inference, I find my intuitions sometimes lead me astray. Additionally, Westreich is one of the most respected researchers in Epi, so perhaps there is something to his comments. Speculating, perhaps his motivation stems from something like the following.
Recall, G-computation recovers a standardised mean $E(Y^a)$ for the potential outcome $Y^a$ when an exposure $A$ is, perhaps contrary to fact, set to some level $Y^{A = a}$, conditional on a vector of covariates $Z$, for an entire population. We assume, but cannot prove, that conditioning on $Z$ will satisfy the exchangeability condition for causal inference, and we furthermore assume that the positivity and consistency conditions for causal inference are also satisfied. Under such assumptions, we may set everyone's level of A to $a=1$, and by simulating responses for a entire population obtain:
$E(Y^1) = E ( E(Y|a = 1, Z ))$
Similarly we may set everyone's level of A to $a=0$, and by simulating responses for the same population obtain:
$E(Y^0) = E ( E(Y|a = 0, Z ))$.
(We may need to compute such expectations as functions...).
This is G-computation: we standardise to the mean distribution of the confounders Z in a population by obtaining the average of the predicted responses at some settings of A = a for an entire population, which we may contrast with another setting A = a* for the same population. (Note we use "standardise" in the epi sense of the term, as balancing the distribution of confounding covariates for the outcome -- there is much scope for terminological muddles here.).
Notably, because the difference in the expectations of the means is the same as the expectations of their difference we may obtain a causal contrast on the difference scale as:
$E(Y^1 - Y^0) = E(Y^1) - E(Y^0)$
This difference is an average causal effect of exposure A = 1 contrasted with the exposure A = 0 on the difference scale. Again, we have computed this causal effect from the differences in the standardised means of the predicted counterfactual outcomes across a population in which, contrary to fact, everyone receives both exposures.
In any case, I find that considerations such as those I have just mentioned help me to build an intuition for why Westreich wants to focus on contrasts of simulated means. Now, what of the computing the variances of the counterfactual responses?
Here I am less confident. The reasoning might be something as follows. It is clear that we cannot apply g-computation directly to model counterfactual variances. This is because $var(Y^1-Y^0)$ is not generally equal to $var(Y^1)$ - $var(Y^0)$, and indeed $var(Y^1 - Y^0)$ is not identified in the data because we do not simultaneously observe $Y^1$ and $Y^0$ on any individual -- a point that Hernan and Robins make in Ch1 and Ch 13 of https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/.
The fact that G-computation and its extensions require special handling for obtaining the variances of potential outcomes might underlie Westreich's quick dismissal of Rubin's rules for partial pooling. Channelling Westreich's line of thought as I have imagined it -- perhaps wrongly -- it is because we only recover contrasts for the means of the simulated potential outcomes that we must turn to bootstrapping for our standard errors (although he mentions, opaquely, closed-form solutions might be possible...)
I will admit that my attempts to build intuition here are something of a stab in the dark. Again, I may be off the mark! I want to underscore I do not have any specific advice. I too am curious. Hopefully, my comments prompt you and others to more considered thoughts.
Thanks again for your time and your wonderful package Vincent!