py-econometrics / pyfixest

Fast High-Dimensional Fixed Effects Regression in Python following fixest-syntax
https://py-econometrics.github.io/pyfixest/pyfixest.html
MIT License
118 stars 27 forks source link

ritest: Increase the performance of the "randomization-t" procedure for RI with OLS #460

Open s3alfisc opened 1 month ago

s3alfisc commented 1 month ago

Context

The current default for randomization inference is the "randomization-c" method, despite evidence that the "randomization-t" procedure is preferable. The main reason for the current default is that the "randomization-c" method is currently much faster. Before making the "randomization-t" the default, I'd like to significantly improve its performance.

Why is the implementation of the "randomization-t" slow, and "randomization-c" fast at the moment?

The reason is that the "randomization-t" procedure currently loops over calls to either feols() or fepois, whereas the "randomization-c" is based on a more optimized numba implementation without all the overhead of the two functions. This is the case because in order to compute the RI t-statistics, we have to compute variance-covariance matrices. For that, we currently have no standalone interface that is numba compatible - the variance covariance matrices are not defined in a standalone, callable function, but are attached to a method of the Feols class.

To do

Blocked by