"All weights are 2143289344" with method = "energy"

[JonNDCN]

Hi, Thanks for the fantastic package.

Method = "energy" seems to work really well in terms of speed, covariate balance and effective sample size in my data.

I am bootstrapping to account for the uncertainty in weights and to generate confidence intervals around my coefficients from the weighted models.

I have encountered an error when using method = "energy" where all weights are found to be equal and I think very large. ("All weights are 2143289344."). A Google of this error suggests this has come up on previous CRAN package checks but Im' not sure.

I can't share my data unfortunately however the error is reproducible with lalonde (this example doesn't have any models just to keep it minimal):

boot.fun <-  function(r, index) {
  w_obj <-
WeightIt::weightit(formula = treat ~ age + educ + race + married + nodegree + re74, estimand = "ATE", method = "energy", data = r[index,])
weights <- w_obj$weights
return(weights)
}

boot(boot.fun, data = lalonde, R = 1000, parallel = "snow", ncpus = 40)  ## Remove parallel and ncpus if you're not able to parallelise.

21 nodes produced errors; first error: All weights are 2143289344

Thank you!

[ngreifer]

Hi Jonathan,

Yes, sadly, that is a limitation of the method. I'm not sure how familiar you are with optimization, but the objective function is nonconvex, meaning it will fail to find a solution sometimes. Other times, it finds a solution without issue. When it fails to find a solution, that error/warning message will appear. If you run the same dataset multiple times with method = "energy", sometimes you will get convergence and other times not. Unfortunately there is no good way around this. When not using bootstrapping, you can rerun the estimation multiple times with the same dataset until a stable solution is found. With bootstrapping, you may have to do something fancy.

One thing you could do is program the bootstrap so that if that warning appears, you could rerun the estimation within the same bootstrap replication, e.g., using tryCatch() or withCallingHandlers(). You could also test to see if all the estimated weights are the same. This is perhaps some advanced programming that you might not want to do, and will increase the time taken to run the analysis.

It may just be that energy balancing is not ready for primetime as a method compatible with bootstrapping. You can try contacting Jared Huling, the inventor of the method, if you have statistical questions about the failure to converge. Unfortunately I can't offer any other help.

[JonNDCN]

Dear Noah, Thank you so much for your incredibly speedy and informative reply. Thanks to your advice I have employed tryCatch with success. I'm sure there's a way to "retry" each optimisation, but for now I'm returning NA when no solution is found using tryCatch, and increasing the number of bootstrap samples to make up for these rare failures.

Thanks again!