search

JuliaML / LossFunctions.jl

Julia package of loss functions for machine learning.
https://juliaml.github.io/LossFunctions.jl/stable
Other
148 stars 34 forks source link

Lipschitz constants for Lipschitz losses ? #148

Open stephanegaiffas opened 2 years ago

stephanegaiffas commented 2 years ago

Hi all and thanks for the this very useful package ! It could be helpful to get in addition to the islipschitzcont attribute, another attribute that gives the value of the global Lipschitz constant, e.g. 2 for your scaling of the L2 loss, 1/4 for the logistic loss, etc. If I am not wrong, this is not featured yet in the package ?

This could be particularly useful for some first-order optimization algorithms in ML, where theoretically optimal learning rates can be automatically tuned using Lipschitz constants (and strong convexity constants as well).

If this is welcomed I could try a PR that implements this.

Best !

juliohm commented 2 years ago

@stephanegaiffas can you draft a PR with a proposal? We will be happy to review it.