Add (binary) hinge loss and multi-class hinge loss

[georgedahl]

This feature request is to add support for the hinge loss, similar to how scikit-learn supports binary and multiclass hinge loss https://scikit-learn.org/stable/modules/generated/sklearn.metrics.hinge_loss.html

For the multiclass version, the one from http://jmlr.csail.mit.edu/papers/volume2/crammer01a/crammer01a.pdf

[mblondel]

I would add binary_hinge_loss, multiclass_hinge_loss, binary_perceptron_loss and multiclass_perceptron_loss. The perceptron loss is similar to the hinge loss but without the margin.

For the binary case, I suggest that we use 0 and 1 labels for consistency with the rest of the losses in JAXopt.

[mblondel]

Here are the formulas:

• binary_perceptron_loss(label, score) = max(0, - score * signed_label)
• multiclass_perceptron_loss(label, scores) = max(scores) - dot(scores, onehot(label))
• binary_hinge_loss(label, score) = max(0, 1 - score * signed_label)
• multiclass_hinge_loss(label, scores) = max(scores + 1 - onehot(label)) - dot(scores, onehot(label))

where signed_label = 2 * label - 1

[froystig]

Closed by @q-berthet in https://github.com/google/jaxopt/commit/60259a486255574a885fbd4fed533574791f087f!