search

dcwuser / metanumerics

Meta.Numerics is library for advanced numerical computing on the .NET platform. It offers an object-oriented API for statistical analysis, advanced functions, Fourier transforms, numerical integration and optimization, and matrix algebra.
http://www.meta-numerics.net
101 stars 28 forks source link

Better initial guesses for logistic regression #15

Open dcwuser opened 6 years ago

dcwuser commented 6 years ago

We do a full-on multi-dimensional optimization to get logistic regression parameters via likelihood maximization. I don't see any alternatives in the literature, but we should at least be able to make a better initial guess to feed into that algorithm than "all zeros", which is what we currently do.

dcwuser commented 6 years ago

For the bivariate case, here is a simple idea I found online. Call x_T and x_F the mean values of x for the true and false cases. Assume \sigma(a + b x_T) = 3/4 \sigma(a + b x_F) = 1/4 Then b = \frac{\sigma^{-1}(3/4) - \sigma^{-1}(1/4)}{x_T - x_F}. This should get the sign right, and the order of magnitude if we are lucky.