beacon-biosignals / Effects.jl

Effects Prediction for Regression Models
MIT License
19 stars 4 forks source link

additional tests of pivoting logic; bootstrapped effects for mixed models #75

Closed palday closed 3 months ago

palday commented 3 months ago

The pivoting logic is very tricky and easy to get wrong, so I added a few more tests to increase our confidence in the results.

I also added an experimental method for computing effects from a MixedModelBootstrap. This may be interesting on its own (though I seriously doubt it will differ much from the vcov-based method because it's a parametric bootstrap), but it also doesn't rely on pivoting to work with rank deficient models and so provides an additional way to cross-check the pivoted vcov-vased method. (The pivoted coefficients are stored in-place as -0.0, so they just cancel out for the prediction itself, but the associated entries in the vcov matrix are NaN, which are poisonous and thus must be pivoted out.)

It also looks like there is a complex compatibility problem between certain Julia, MixedModels and StatsModels 0.6 versions, which becomes apparent in the use of the bootstrap. I'm just going to drop explicit StatsModels 0.6 testing. We still test against it via the Julia 1.6 test, which limits the MixedModels version to a version requiring StatsModels 0.6. Otherwise, we can backport bugfixes to the 1.2 series and StatsModels 0.7 is well supported nowadays.

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.53%. Comparing base (d5d8fbb) to head (9c26501). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #75 +/- ## ========================================== + Coverage 99.45% 99.53% +0.07% ========================================== Files 6 6 Lines 185 214 +29 ========================================== + Hits 184 213 +29 Misses 1 1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.