search

scikit-adaptation / skada

Domain adaptation toolbox compatible with scikit-learn and pytorch
https://scikit-adaptation.github.io/
BSD 3-Clause "New" or "Revised" License
60 stars 16 forks source link

[MRG] Add TarS method #93

Closed antoinecollas closed 7 months ago

antoinecollas commented 7 months ago

This PR implements the TarS method from http://proceedings.mlr.press/v28/zhang13d.pdf (mainly eq.6). Related issue: #43

codecov[bot] commented 7 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 92.05%. Comparing base (8a46819) to head (5681f20).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #93 +/- ## ========================================== + Coverage 91.86% 92.05% +0.18% ========================================== Files 43 43 Lines 3392 3485 +93 ========================================== + Hits 3116 3208 +92 - Misses 276 277 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

antoinedemathelin commented 7 months ago

Hello, Yes, solving constrained QP is possible with scipy link. There are three available solvers for that: trust-constr, SLSQP and COLYBA. I tested them, and SLSQP is the fastest. Giving the Jacobian (computed analytically) fasten the computation also.

Indeed, CVXOPT is faster (perhaps 10 times faster than scipy for 1000 samples). However, I do not recommend including it in the requirements. I did it for adapt, but run in multiple bugs after new cvxopt releases...

My suggestion is to provide a faster optimization solver, using the Frank-Wolfe algorithm, which can be easily implemented. Frank-Wolfe works well for weighting methods based on MMD or KL minimization, such as KLIEP and KMM. It really speeds up the optimization. cf this paper : https://webdocs.cs.ualberta.ca/~dale/papers/ijcai15.pdf . It could be proposed as an alternative for the user via a solver argument, to keep track of the original algorithm that uses classic QP solvers.

antoinecollas commented 7 months ago

Thanks for all the infos @antoinedemathelin !