search

amplab / keystone

Simplifying robust end-to-end machine learning on Apache Spark.
http://keystone-ml.org/
Apache License 2.0
468 stars 116 forks source link

WeightedBlockSolver for multiple lambdas #229

Open ericmjonas opened 8 years ago

ericmjonas commented 8 years ago

We often perform a sweep over lambdas to figure out the right regularization -- it matters a lot.

@shivaram mentioned that it's possible for the weighted solver to potentially solve for a large number of lambdas with minimal overhead. There was evidently some code in a repo associated with a paper that is an example of this. It might be possible (per Shiv's suggestion) that @tomerk could hack on this?

shivaram commented 8 years ago

I misremembered things a little bit - we've never done multi-lambda for the old BlockWeightedLeastSquares. But for the new PerClassWeightedLeastSquares it should be pretty simple to add it by just following wherever we have a loop around lambdas in the unweighted implementation [1].

FWIW If we want feature compatibility across the solvers, the other one should also be possible -- but my guess is that it'll not be that cheap in terms of sharing costs and need some new code.

[1] https://github.com/amplab/ml-matrix/blob/c0c0d7c754f3873c419af85bc298cdde2cb4cdfe/src/main/scala/edu/berkeley/cs/amplab/mlmatrix/BlockCoordinateDescent.scala#L88