ngreifer / WeightIt

WeightIt: an R package for propensity score weighting
https://ngreifer.github.io/WeightIt/
102 stars 12 forks source link

No check in ebal method to see if optim converged #33

Closed ntopousis closed 1 year ago

ntopousis commented 2 years ago

Hi, firstly, thank you for building and maintaining this package. Want you to know that I really appreciate your contributions.

I've been recently running into an issue trying to port entropy balancing from R to python. Where, when I create entropy balancing weights (EBWs) for each cohort in my data, I get errors (in python) for some cohorts. I then ran Jens Hainmueller's ebal package cohort-by-cohort on the same data and replicated the error. However, I do not get such an error when I use WeightIt to create EBWs by cohort on the same dataset.

I believe the issue is that weightit uses optim(method = "BFGS") to solve the optimization problem necessary to create entropy balancing weights; however, there is no check to ensure that the optimization actually converged. Optim returns weights even if it did not converge and also provides a value optim$convergence to indicate whether or not the optimization problem converged successfully.

I see that we can ask WeighIt to return the output of the call to optim and check ourselves whether or not entropy balancing converged. I can implement that on my end going forward. I do think that including some sort of warning if the optimization does not converge, or, as a default, erroring if the optimization does not converge could make this package more friendly for less technical users who assume that if the package returns an output without warnings or errors then everything went smoothly.

ngreifer commented 1 year ago

Almost a year later, but I wanted to let you know I did add this thanks to your suggestion. When entropy balancing fails to converge, weightit() now throws a warning.

ntopousis commented 1 year ago

Nice!