search

linkedin / photon-ml

A scalable machine learning library on Apache Spark
Other
792 stars 185 forks source link

Disable L2 Regularization on Intercept #440

Closed ashelkovnykov closed 5 years ago

ashelkovnykov commented 5 years ago

This is continuation of the work started by @biyan-linkedin in PR #439. Thank you for your work Bill!

Previously, Photon would regularize the entire coefficient vector, including the intercept coefficient if one was present. One of our users had this to say:

Penalizing intercept makes the behavior of L2 regularization depends
on the pos/neg ratio, and often leads to hard-to-interpret intercept.
Especially when we have a lot of training data and most of the features
are extremely sparse.