benbhansen-stats / propertee

Prognostic Regression Offsets with Propagation of ERrors, for Treatment Effect Estimation (IES R305D210029).
https://benbhansen-stats.github.io/propertee/
Other
2 stars 0 forks source link

Transform `PreSandwichLayer` `offset` object to a `SandwichLayer` during `lmitt.lm()`/`as.lmitt()` #123

Closed jwasserman2 closed 1 year ago

jwasserman2 commented 1 year ago

dmod1 and dmod2 below do not result in the same variance calculations because dmod2$model$(offset) isn't a SandwichLayer object--offst2 was never converted from a PreSandwichLayer during the as.lmitt process. Therefore, it has no keys slot, and the variance sample size denominators and functions for ordering observations in estfun outputs get thrown off. As one can see with dmod3, a fix would be to convert offst2 to a SandwichLayer during .convert_to_lmitt() since a necessary Design object is provided.

> data(simdata)
> copy_simdata <- simdata
> copy_simdata$schoolid <- seq(900, 949)
> C_df <- rbind(copy_simdata[, c("schoolid", "x", "y")], data.frame(schoolid = seq(1000, 1049), x = rnorm(50), y = rnorm(50)))
> cmod <- lm(y ~ x, C_df)
> des <- rct_design(z ~ uoa(schoolid), copy_simdata)
> lm1 <- lm(y ~ assigned(), copy_simdata, weights = ett(design = des), offset = cov_adj(cmod, NULL, NULL, "schoolid"))
> dmod1 <- lmitt(lm1, design = des)
> vcovDA(dmod1)
            (Intercept)  assigned()
(Intercept)  0.02766383 -0.03963726
assigned()  -0.03963726  0.08796185
attr(,"type")
[1] "CR0"
> 
> wts2 <- ett(des, data = copy_simdata)
> offst2 <- cov_adj(cmod, copy_simdata, by = "schoolid")
> lm2 <- lm(y ~ assigned(), copy_simdata, weights = wts2, offset = offst2)
> dmod2 <- lmitt(lm2, design = des)
> vcovDA(dmod2)
            (Intercept)  assigned()
(Intercept)  0.04425449 -0.04425449
assigned()  -0.04425449  0.09125442
attr(,"type")
[1] "CR0"
> 
> wts2 <- ett(des, data = copy_simdata)
> offst2 <- cov_adj(cmod, copy_simdata, design = des, by = "schoolid")
> lm3 <- lm(y ~ assigned(), copy_simdata, weights = wts2, offset = offst2)
> dmod3 <- lmitt(lm3, design = des)
vcovDA(dmod3)
            (Intercept)  assigned()
(Intercept)  0.02766383 -0.03963726
assigned()  -0.03963726  0.08796185
attr(,"type")
[1] "CR0"
josherrickson commented 1 year ago

One thing I never really filled out was the validity check for DirectAdjusted objects. Should we put this in there? If the embedded lm has an offset, it cannot be PreSandwichLayer? (We don't want to enforce it being only SandwichLayer for the obvious reasons.)

Alternatively if we don't need to be so strict, should we give a warning or message in show.DirectAdjusted or print.summary.DirectAdjusted? I suspect not as "SandwichLayer" and "PreSandwichLayer" won't mean much to most users. Unless there's an easy way to frame the message as a fix rather than an issue. Maybe "pass a design= argument to cov_adj()"?

jwasserman2 commented 1 year ago

In .convert_to_lmitt, I think we should check whether the offset argument (either in a lmitt.formula call or an lm passed in) is a PreSandwichLayer, and if it is, then we call as.SandwichLayer on it with the design argument of that function. A Design object must be passed in to either lmitt.formula or as.lmitt()/lmitt.lm(), so we can always make this conversion

josherrickson commented 1 year ago

I chose not to enforce no PreSandwichLayer in DirectAdjusted objects right now; we can always add this later.

> data(simdata)
> copy_simdata <- simdata
> copy_simdata$schoolid <- seq(900, 949)
> C_df <- rbind(copy_simdata[, c("schoolid", "x", "y")], data.frame(schoolid = seq(1000, 1049), x = rnorm(50), y = rnorm(50)))
> cmod <- lm(y ~ x, C_df)
> des <- rct_design(z ~ uoa(schoolid), copy_simdata)
> lm1 <- lm(y ~ assigned(), copy_simdata, weights = ett(design = des), offset = cov_adj(cmod, NULL, NULL, "schoolid"))
> dmod1 <- lmitt(lm1, design = des)
> vcovDA(dmod1)
            (Intercept)  assigned()
(Intercept)  0.02934713 -0.04025267
assigned()  -0.04025267  0.08825856
attr(,"type")
[1] "CR0"
> wts2 <- ett(des, data = copy_simdata)
> offst2 <- cov_adj(cmod, copy_simdata, by = "schoolid")
> lm2 <- lm(y ~ assigned(), copy_simdata, weights = wts2, offset = offst2)
> dmod2 <- lmitt(lm2, design = des)
> vcovDA(dmod2)
            (Intercept)  assigned()
(Intercept)  0.02934713 -0.04025267
assigned()  -0.04025267  0.08825856
attr(,"type")
[1] "CR0"
> wts2 <- ett(des, data = copy_simdata)
> offst2 <- cov_adj(cmod, copy_simdata, design = des, by = "schoolid")
> lm3 <- lm(y ~ assigned(), copy_simdata, weights = wts2, offset = offst2)
> dmod3 <- lmitt(lm3, design = des)
> vcovDA(dmod3)
            (Intercept)  assigned()
(Intercept)  0.02934713 -0.04025267
assigned()  -0.04025267  0.08825856
attr(,"type")
[1] "CR0"