Add coverage metric and relevant figure to output

[bblette1]

Added some parallel results for a coverage metric. Also reduced n from 1000000 to 100000 because my local computing is poor, but you can undo that and rerun on your end.

[LucyMcGowan]

I think an exploration of coverage definitely makes sense although I'm wondering if using the naive intervals from the outcome model is quite fair since we know those will definitely be wrong for the weighted model. I think we'll see poor coverage either way (was just reading this: https://onlinelibrary.wiley.com/doi/epdf/10.1002/bimj.202000267 and it seems basically all variance estimation fails for the GPS weighted estimates).

Now that I am writing this I wonder if we should look at variance rather than coverage since the bias issue may be squashing the coverage issue (i.e. since it is so biased it will have terrible coverage just from that rather than from the lack of correct variance estimation?) Although I guess most practitioners would just hope that the coverage would include any potential bias 🤔

This whole exercise has made me wonder if we should ever be fitting these propensity score weights for continuous exposures -- I'm started to prefer a stratified approach 😬

[bblette1]

That's a fair point, although usually not accounting for the weights means the standard error is conservative, such that there is overcoverage. And since this had undercoverage, I think accounting for weight estimation would only make it look even worse? But it might not be guaranteed.

That's a good point, I think you're right that the undercoverage is mostly a function of bias. Some people have proposed "bias-centered coverage" as a way around this, but to be honest I haven't ever used that in a paper. So let's focus on just variance. In general, please feel free to reject any code/writing I try to push!!

Yes, most of my dissertation involved continuous exposures, and the number 1 thing I tell people since then is to avoid continuous exposure causal inference whenever possible lol, especially for weighting approaches. The assumptions are really hard to fulfill. G-formula is a bit cleaner and is my preference in continuous applications now

[LucyMcGowan]

Oh this is such a good point! I guess in this particular case since I don’t have a heterogeneous treatment effect the “covariates adjustment” in my plots is exactly equal to the g-formula estimate, maybe I should label as that. And now that you mention it maybe that is my big take away 😅 why would you ever pick weighting over g comp? Maybe if you knew the functional form of the propensity score better? But here we are specifying it exactly and it is still failing spectacularly!

[bblette1]

Oh good point, I guess the code is already doing g-computation. Then maybe a better narrative overall would be: Even though propensity score weighting and g-computation both assume positivity, when the exposure is continuous, propensity score weighting seems much more sensitive to that assumption?

Yeah I suppose the main use case would be if you're very confident in the propensity model but not the outcome model, and you also think any positivity violation is mild. We could probably construct that scenario in the sims using a mis-specified adjusted outcome model and positivity similar to your top panel from the sims. But probably not a common scenario in practice.

On Fri, Nov 18, 2022 at 3:38 PM LucyMcGowan @.***> wrote:

Merged #1 https://github.com/LucyMcGowan/writing-positivity-continous-ps/pull/1 into main.

— Reply to this email directly, view it on GitHub https://github.com/LucyMcGowan/writing-positivity-continous-ps/pull/1#event-7847273321, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3UNIYQXF6IIZJVOIMVZB3WI7SNFANCNFSM6AAAAAASAOO4UE . You are receiving this because you authored the thread.Message ID: <LucyMcGowan/writing-positivity-continous-ps/pull/1/issue_event/7847273321 @github.com>