Optimize drop_singletons!()

FixedEffects / FixedEffectModels.jl

Fast Estimation of Linear Models with IV and High Dimensional Categorical Variables

Other

225 stars 46 forks source link

Closed droodman closed 7 months ago

droodman commented 7 months ago

I'm new to PRs so feedback welcome.

This PR tweaks drop_singletons!() for speed, mainly by:

Multi-threading the second for loop, which drops the singletons
Stopping the count of observations in each FE group at 2, and storing the counters in bytes, which reduces allocations and probably uses CPU caches more efficiently

matthieugomez commented 7 months ago

Thanks! Did you do any test that it tends to accelerate things?

Can you please remove your uneeded changes to Manifest?