Ordinalizing Margin Losses

[mihirparadkar]

Right now LowRankModels implements an OrdinalHingeLoss which is related to the HingeLoss. However, as we port the losses over to LossFunctions, I was thinking that this ordinalization is not unique to the HingeLoss, but could be used for any margin loss (LogisticLoss, SquaredHingeLoss, etc.). The Rennie paper that describes the ordinal hinge loss ( http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.124.9242 ) seems to confirm this.

Perhaps we should reimplement it generically across margin losses by using a wrapper type a la ScaledLosses from LossFunctions (I'm thinking OrdinalMarginLoss{T<:MarginLoss, min, max} or something like that).

[mihirparadkar]

I've started writing code to do this for arbitrary margin losses, but I think the loop for explicit summation is necessary. I don't think the clever tricks in OrdinalHingeLoss generalize to other losses like squared hinge or logistic. However, since the number of ordinal levels should usually be relatively small ( <10 or so), the loop could be optimized by unrolling, which is implemented in Unroll.jl as a macro.

[madeleineudell]

Cool!

On Mon, Jun 26, 2017 at 7:20 AM, mihirparadkar notifications@github.com wrote:

I've started writing code to do this for arbitrary margin losses, but I think the loop for explicit summation is necessary. I don't think the clever tricks in OrdinalHingeLoss generalize to other losses like squared hinge or logistic. However, since the number of ordinal levels should usually be relatively small ( <10 or so), the loop could be optimized by unrolling, which is implemented in Unroll.jl as a macro.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/madeleineudell/LowRankModels.jl/issues/80#issuecomment-310966124, or mute the thread https://github.com/notifications/unsubscribe-auth/AAyp9AnHuPs-l5yA0PeqkANb6fON_M4pks5sHz-pgaJpZM4OEt4X .

-- Madeleine Udell Assistant Professor, Operations Research and Information Engineering Cornell University https://people.orie.cornell.edu/mru8/ (415) 729-4115

[mihirparadkar]

I opened the PR, but even with ~5 levels, the OrdinalMarginLoss(HingeLoss(), 5) is 6 times slower than an ordinary MarginLoss when applied to an array. I think loop unrolling using @nexprs will definitely help a lot, especially because ordinal losses usually don't have a lot of levels.

Of course, in order to allow this unrolling, I have to include the number of levels as a type parameter instead of as a field, and then use @generated along with @nexprs so I can get that unrolling at compile-time.