kjunelee / MetaOptNet

Meta-Learning with Differentiable Convex Optimization (CVPR 2019 Oral)
Apache License 2.0
524 stars 97 forks source link

Why the SVM implemented by yourself is better than that in sklearn #34

Closed DamonAtSjtu closed 4 years ago

DamonAtSjtu commented 4 years ago

Some external library has already implemented the SVM algorithm, e.g. sklearn. And I think sklearn's SVM algorithm is also implemented through the dual form. Why don't you just use the SVM in sklearn. What is the advantages of your own implemented SVM or what's special about it?

kjunelee commented 4 years ago

Note that the core component of our method is backpropagating through the SVM solver in order to learn representations that generalize better when used with Crammer & Singer's SVM objective (Eq 4 in our paper). As you mentioned, one can use off-the-shelf SVM solvers e.g. sklearn, and then compute Eq 8 by plugging in the solution from sklearn. OptNet (Amos & Kolter ICML 2017) shows for the case of quadratic programs, one can compute Eq 8 for "almost free" by leveraging derivatives computed during primal-dual interior point method steps. Our implementation uses OptNet framework to efficiently compute Eq 8.