carloscinelli / sensemakr

Suite of sensitivity analysis tools for OLS
https://carloscinelli.com/sensemakr/
83 stars 15 forks source link

Implement support for fixest::feols estimation results #53

Closed grlju closed 1 year ago

grlju commented 1 year ago

Hi team,

Thank you very much for an amazing package!

I have been working on extending the package to make it compatible with the fixest::feols estimation command. I believe that this will make the package even more useful for a range of analyses. I believe that this extension of the package would allow you to address the feature request for plm or lfe (issues/43) and would allow R users to address this issue: issues/50.

You will see that I have labeled this version 0.1.5 since it would be quite a substantive update, and I have added myself as a contributor. Of course, I am happy to discuss this.

If you have any questions about any of the code, please don't hesitate to let me know.

Kind regards, Greg

carloscinelli commented 1 year ago

Hi @grlju thanks, this is great. I'm checking some of the details and made some fixes.

As a benchmark, all results with lm as in:

model_lm <- lm(peacefactor ~ directlyharmed + age + farmer_dar + herder_dar + pastvoted + hhsize_darfur + female + village, data = darfur)

Should match exactly the results run with feols as in:

model <- fixest::feols(peacefactor ~ directlyharmed + age + farmer_dar + herder_dar + pastvoted + hhsize_darfur + female | village, data = darfur)

There were some discrepancies between the two, and the main culprit was the degrees of freedom.

The command: degrees_freedom(model, "t") does not return the correct residual degrees of freedom, the correct command is degrees_freedom(model, "resid", vcov = "iid").

I fixed that through out the code, and also adjusted the testing (thanks for implementing all tests!).

Also, regarding robust standard errors, we need to separate two parts: computing R2s and adjusted estimates, versus adjusting the standard error itself.

For computing R2 and adjusting estimates, we always use the "iid" standard error. That's because the SE is being used simply as a computational shortcut, to recover the residual standard deviation of Y and D. Thus, even if one uses robust SEs for inference, the adjusted estimate uses the vanilla SE (or we can use the residual standard deviations directly).

For the standard errors themselves, then we can use the robust standard error --- however, to properly do that, we need to use the delta method. This is not hard to implement, but since it's not done yet, for now I am including a message to users in case they pass a model with a non-standard vcov.

carloscinelli commented 1 year ago

@grlju PS: could you double check my changes?

grlju commented 1 year ago

Thanks so much for the fixes!

I was not quite sure about the right degrees_freedom option to use (you may have seen it in the iterations of the commits). You are right. The differences are due to using "t" rater than "resid" in the degrees_freedom function.

I have also checked your changes and run all the tests, and everything works fine, including the note about se's.

chadhazlett commented 1 year ago

Thanks everybody -- this is a great addition!

On Mon, Aug 1, 2022 at 10:25 PM Carlos Cinelli @.***> wrote:

Merged #53 https://github.com/carloscinelli/sensemakr/pull/53 into master.

— Reply to this email directly, view it on GitHub https://github.com/carloscinelli/sensemakr/pull/53#event-7106118139, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKUFSGZWRSGH4H4U2AFYTTVXCBJJANCNFSM55F3JUOQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

maswiebe commented 1 year ago

It looks like the reference page is not updated to include instructions for feols.

carloscinelli commented 1 year ago

@maswiebe thanks, we will update the website.