search

JuliaML / META

Discussions related to the future of Machine Learning in Julia
MIT License
10 stars 2 forks source link

Penalties.jl? #12

Closed joshday closed 8 years ago

joshday commented 8 years ago

Maybe it's because I'm unsure of what's going into ObjectiveFunctions.jl, but do we want a Penalties.jl? My thinking is that penalty implementations are rather straight-forward and we could get something into Metadata sooner than if they are attached to ObjectiveFunctions, which may take more thought to get right.

Side note: My summer work is ending, so I'll actually start contributing soon. I'd be happy to work on penalties, wherever they live.

Evizero commented 8 years ago

sounds good to me! I like the small-package-approach.

This is the best interface I think we had so far (which was very heavily based on your implementation with some small changes) was this (disregard the strange inlines - i tested something):

https://github.com/JuliaML/Losses.jl/blob/8f877395c9e46e7232753704b4e686e672b35267/src/loss/abstract.jl#L232-L257 https://github.com/JuliaML/Losses.jl/blob/8f877395c9e46e7232753704b4e686e672b35267/src/loss/params.jl#L45-L71

We since removed all penalty code when we decided to specialize the package on losses only. I very much liked what I saw back then and would love your input on penalties

I think my main change was the introduction of len. The idea behind len is that it allows one to control if the intercept is penalized or not. (turns out with SVMs you may want to go in either direction depending on the guarantees you are looking for)

ahwillia commented 8 years ago

The ObjectiveFunctions repo needs direction. I say go ahead and start making Penalties and I'll try to sync up. Maybe we don't need ObjectiveFunctions, but we need some principled way to combine Losses and Penalties into a model to be optimized.

Evizero commented 8 years ago

The ObjectiveFunctions repo needs direction.

Yes, well, I think we are making great progress by solving one problem at a time. Thanks to your and Tom's continuous input, the Losses.jl package is in a pretty good shape already and we can slowly shift our focus on other problems that have to be solved. It seems to me that penalties might be a good next step.