search

scikit-learn-contrib / metric-learn

Metric learning algorithms in Python
http://contrib.scikit-learn.org/metric-learn/
MIT License
1.4k stars 233 forks source link

Algorithm wish list #13

Open perimosocordiae opened 8 years ago

perimosocordiae commented 8 years ago

Methods that we haven't implemented yet, but would like to. In no particular order:

bhargavvader commented 8 years ago

Hello, any tips or further documentation on how to go about doing this? And which one of these would be highest priority?

perimosocordiae commented 8 years ago

I don't personally have a priority, but if you want to rank them you could look at their citation counts on Google Scholar. That should give you an idea of what's more popular in the literature. I would start with one that seems easy to implement, though.

Each method should have its own Python file in the metric_learn directory, which defines a class that inherits from BaseMetricLearner. This subclass should implement a .fit() method, as well as one of the .metric() or .transformer() methods. As a really simple example, you can start with the Covariance class.

perimosocordiae commented 8 years ago

Once you have something that works, add a class to test/metric_learn_test.py which exercises the new algorithm. You should also verify that your implementation matches the reference code, if possible.

bhargavvader commented 8 years ago

Thanks for all the details! The paper by Xing, et al is I think the most popular (and one of the first in Metric Learning) algorithm/paper among the list, I'll give it a shot.

devashishd12 commented 8 years ago

Hey @perimosocordiae can I take up the MLKR implementation? I was thinking of using the transformer() method to use MLKR for dimensionality reduction and hence return X_new as the output. Also, I was thinking of having the A matrix stored as an attribute after the fit() operation thus enabling the user to access it independently also if needed. Does this seem fine to you? I'll take the output dimension as a parameter for __init__() to achieve the extra optimisation as done in the matlab code.

perimosocordiae commented 8 years ago

@dsquareindia Sure, MKLR is a good choice. The transformer() method should return the matrix A (as defined the paper), and the transform() method will take care of the dimension reduction without needing to be overridden. In this way, the base class metric() method will also work as intended (corresponding to M in the paper).

aschuman commented 7 years ago

KISSME metric learning would be a nice addition (no optimization). You can find a reference Matlab implementation and the paper here: https://lrs.icg.tugraz.at/research/kissme/ There is also a python implementation (which I have not verified) here: https://github.com/Cysu/dgd_person_reid/blob/master/eval/metric_learning.py

perimosocordiae commented 7 years ago

@aschuman thanks, I've added it to the list.

anirudt commented 7 years ago

LDML

RishabGoel commented 7 years ago

hi @perimosocordiae I would like to take up Probabilistic Global Distance Metric Learning. thanks :)

perimosocordiae commented 7 years ago

@anirudt Thanks, added.

@RishabGoel Go for it! I look forward to your pull request.

bhargavvader commented 7 years ago

Hello, just wanted to know where the short form PGDM came up - I can't seem to find it in the paper linked (which is the paper just called "Distance metric learning, with application to clustering with side-information"), by Xing et al.

RishabGoel commented 7 years ago

@bhargavvader Probably the wrong link. The metric is a non probabilistic one in the link. But find the PRobabilistic one in the following link: http://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf I am bit busy right now so I am not able to do much, but will submit a PR soon. Hope this helps! Regards

perimosocordiae commented 7 years ago

@bhargavvader, @RishabGoel: yep, that's correct. I pulled the citation from the survey paper without looking too closely at it. I'll fix the link now.

Callidior commented 7 years ago

The method by Xing et al. (2003) is also sometimes referred to as "MMC", e.g., in the LMNN paper by Weinberger et al. I guess, "MM" stands for "Mahalanobis Metric Learning" and "C" for "Clustering".

souravsingh commented 7 years ago

@perimosocordiae I would like to take up DistBoost and KernelBoost

perimosocordiae commented 7 years ago

@souravsingh Great! Feel free to open a PR whenever you'd like to get feedback. I just noticed the paper links for those methods are broken, so if you find working links to the papers let me know.

souravsingh commented 7 years ago

@perimosocordiae I think these are the links to the papers-

DistBoost- http://www.cs.huji.ac.il/~daphna/papers/distboost-icml.pdf KernelBoost- http://leibniz.cs.huji.ac.il/tr/919.pdf

I couldn't find the reference implementations accompanying these algorithms. There is an implementation for DistBoost here- http://www.scharp.org/thertz/code.html

perimosocordiae commented 7 years ago

@souravsingh thanks, I've updated the list with your links.

bellet commented 5 years ago

Another suggestion: the triplet-based method proposed in the paper below (heavily cited, very useful for information retrieval applications): https://www.cs.cornell.edu/people/tj/publications/schultz_joachims_03a.pdf

Given that we currently do not have any triplet-based weakly supervised approach I think we could make this a priority. A GD or SGD implementation should not be very difficult

hansen7 commented 5 years ago

How about add the extension to the existed LSML algorithm, in the original paper and code, there is the extension of the vanilla algorithm called spLSML, which seems to be is just to add a L1 norm of the learned Mahalanobis matrix in the loss function, thus with a novel gradient update method, it should be quick to implement.