search

ocramz / sparse-linear-algebra

Numerical computation in native Haskell
GNU General Public License v3.0
88 stars 10 forks source link

Application: Generalized linear models #63

Open ocramz opened 6 years ago

ocramz commented 6 years ago

References :

adamConnerSax commented 5 years ago

Just a note here. I've been working my way through replicating some of Rs lmer and glmer using sparse-linear-algebra as the sparse matrix representation. It's going slowly but going.
I'm using Cholmod (via hcholmod and some extras I added) and nlopt (via hmatrix-nlopt). Right now I'm pretty close to a working version for lmer and it's done in the same way that R does it which should make generalizing to glmer more or less straightforward. The Cholmod interface needs some work, and when I come up for air, I plan to build a cholmod-sla package to give useful access to cholmod routines using sparse-linear-algebra on the haskell side. That's fairly straightforward and hcholmod needs replacing anyway since, though it's super useful, it doesn't expose enough, doesn't work with a nice sparse matrix representation and, I think, isn't handling all the FFI stuff quite right. Though I'm not sure about that last bit. Another issue for later is getting the problem specification types correct. For now I've just rolled something that works for specifying the grouping of observations and which random-effects to model. But that could probably benefit from some more eyes at some point. Anyway, it's most definitely a WIP but you can see what's going on so far here: https://github.com/adamConnerSax/glm-haskell which is more-or-less, following the (lme) algo as described here: https://arxiv.org/pdf/1406.5823.pdf Then I will move onto the glm bits from here (which use the same pieces but with another optimization step in the middle to handle the weights coming from the link function): https://cran.r-project.org/web/packages/lme4/vignettes/Theory.pdf

Once this all (the bare bones of lmer and glmer) works, I'd be happy to see what we can do about adding different algos, like whatever is in glmnet. I've been focused on glmer because it is what's used in some papers doing electorate modeling and I have some plans to explore that work once I have the tools.

ocramz commented 5 years ago

Wow, thanks for sharing your work! I agree the FFI stuff is pretty confusing; I've never looked into the cholmod internals. However I'd be happy to support you in using/extending SLA.

adamConnerSax commented 4 years ago

Just an update here: I've got LMMs and GLMMs (with Normal, Bernoulli, Binomial, Poisson, and Gamma distributions for observations) working, at least roughly. I'm working on some confidence interval stuff--parametric bootstrapping and likelihood profiling. All following lme4. You can see that work on this branch: https://github.com/adamConnerSax/glm-haskell/tree/GLMMs The interface and internals are sort of terrible at this point. I need to use it more on real things to see how to simplify its use. As always, feedback and thoughts are most welcome!