search

scikit-learn-contrib / lightning

Large-scale linear classification, regression and ranking in Python
https://contrib.scikit-learn.org/lightning/
1.73k stars 214 forks source link

FTRL solver #57

Open binga opened 8 years ago

binga commented 8 years ago

Hello,

Are there any plans of including FTRL solver to this library? I guess it's fairly straight forward. I'm sorry I might not be able to contribute it myself because I am not fluent in Cython etc. Reference:

  1. https://www.eecs.tufts.edu/~dsculley/papers/ad-click-prediction.pdf
  2. https://www.kaggle.com/c/avazu-ctr-prediction/forums/t/10927/beat-the-benchmark-with-less-than-1mb-of-memory

Thanks!

mblondel commented 8 years ago

What's your motivation?

binga commented 8 years ago

For large datasets, we typically tend to use online learning and FTRL is known to do well because of the adaptive learning rate scheme and it's ability to activate a co-ordinate only after a category has been seen a good number of times in training data. In many competitions on Kaggle, I have seen FTRL doing far better than SGD and I believe it's a good algorithm to add to this library!

mpearmain commented 8 years ago

+1 for FTRL

CasyWang commented 7 years ago

+1 for FTRL

gom7745 commented 7 years ago

+1 for FTRL @mblondel May I try this?

sergei3000 commented 7 years ago

+1 for FTRL

SimpleJian commented 7 years ago

+1 for FTRL

mblondel commented 7 years ago

You can give as many +1 as you want, FTRL is not gonna implement itself :b

DachuanZhao commented 4 years ago

Any processing now ?