search

X-DataInitiative / tick

Module for statistical learning, with a particular emphasis on time-dependent modelling
https://x-datainitiative.github.io/tick/
BSD 3-Clause "New" or "Revised" License
480 stars 105 forks source link

Objective function formulation for Logistic Regression #489

Open fabian-sp opened 2 years ago

fabian-sp commented 2 years ago

Hi all,

I can not find the exact formulation of the objective function that is minimized in LogisticRegression. In particular, how exactly does C appear in the objective (e.g. if I want to do l1-penalization)?

Thank you!

achab commented 2 years ago

The parameter C is the inverse of the regularization strength. It's a convention also used by scikit-learn, see.

fabian-sp commented 2 years ago

Yes, but in scikit-learn the loss function values are not averaged. I believe that in tick you solve:

min 1/C ||x||_1 + 1/n \sum_{i=1}^n log(...)

achab commented 2 years ago

Indeed, there is a scaling difference

fabian-sp commented 2 years ago

Thanks! Another question I would have is: how can I set/access the starting point of a solver? So when using SVRG for logistic regression, what is the default starting point?

stephanegaiffas commented 2 years ago

Cf https://x-datainitiative.github.io/tick/modules/solver.html You can choose the one you want using the optim api otherwise it's zero I think (long time not using it)