WeightIt method=gbm problem

[Geoff-Kahn]

I am attempting to assess the covariate balance following propensity score weighting implemented in the WeightIt package. When I use method = "ps" in WeightIt, everything functions normally. However, when I use method = "gbm" and try to assess the balance with bal.tab I get the following love plot:

as well as the warning message, "Warning message: Large mean differences detected; you may not be using standardized mean differences for continuous variables."

I tried to make sure standardized differences were being used with set.cobalt.options(binary = "std", continuous = "std"), but this did not resolve the problem. The difference in prop.score does seem to be sensitive to the stop.method used, but in all cases it's still way larger than I would expect. I'm not sure what else to try, and would greatly appreciate any advice. My code is below, but it's all standard stuff, and again, it works fine when method = "ps" so I'm not sure what's going on here. Thanks much.

weight.gbm <- weightit(RCTflag ~ Urbanicity + Region + GradRate + StTchRatio + TotEnroll + FARMS + StudentN + Grade10 + Grade11 + Grade12, data=mydata.complete, method="gbm", estimand="ATT", stop.method="es.mean") weight.balance <- bal.tab(weight.gbm, un = TRUE) weight.balance love.plot(weight.balance, thresholds = .25, title="GBM weighting")

[ngreifer]

My guess is that what you observing is the truth and not an error. It might be caused by the estimated propensity score having a very small standard deviation in the treated group, which makes the SMD look huge. Because GBM creates a categorical rather than smooth propensity score due to the binning, this is something that can happen. You can remove the propensity score from the plot using drop.distance = TRUE. The propensity score should not be used to assess balance anyway.

[Geoff-Kahn]

Thanks very much!