[FEA] Expose FAISS New NearestNeighbors metrics

[cjnolet]

FAISS recently exposed several important distance metrics through their brute force kNN interface that will be very useful for cuml.

These metrics can be exposed directly on NearestNeighbors and variants using the “metric” argument. They can also be exposed in UMAP using the “metric” and “target_metic” arguments.

[cjnolet]

Linking https://github.com/facebookresearch/faiss/issues/848

Some notes about support for the rest of the scipy distances:

Base metrics supported "out of the box" by FAISS:

• METRIC_INNER_PRODUCT
• METRIC_L1,
• METRIC_Linf
• METRIC_Lp metrics are supported (includes use of Index::metric_arg (C++))
• METRIC_Canberra
• METRIC_BrayCurtis
• METRIC_JensenShannon are available as well

Metrics that can be supported based on above or with light preprocessing:

• [x] "cosine" is equivalent to inner product of vector angles (eg. normalize out magnitudes)
• [x] "correllation" equivsalent to cosine of mean centered vectors
• [x] "cityblock" equivalent to L1
• [x] "euclidean" equivalent to L2
• [ ] "mahalanobis" equivalent to L2 of the mean centered matrix * the inverse of the covariance (cupy supports cholesky so this shouldn't be too bad to do.
• [x] "minkowski" - Equivalent to Lp
• [x] "sqeuclidean" - Euclidean with squared outputs (default FAISS output)
• [ ] "seuclidean" - equivalent to euclidean of mean centered and unit variance vectors
• [ ] "wminkowski" - equivalent to minkowski w/ an array of weights applied column-wise.

In generall, we probably want to give the user the option of using a copy of the input data for preprocessing or modifying the data in place. This will also extend the single GPU implementation to the Dask environment, where copying could be expensive enough to be prohibitive but we can't assume the user is okay modifying the data in place.

A good example of mahalanobis preprocessing here: https://gist.github.com/mdouze/6cc12fa967e5d9911580ef633e559476

[cjnolet]

Cosine

preprocess = cp.sum(X, axis=1)
X /= preprocessing
inds, dists = kneighbors(X)
X *= preprocessing

Correllation

preprocessing = cp.mean(X, axis=1)
X -= preprocessing
inds, dists = cosine(X)
X += preprocessing

[JohnZed]

Will add new metrics but follow a similar pattern to: https://github.com/rapidsai/cuml/pull/2394

[JohnZed]

Most important are done for 0.15. Additional to come in 0.16.

[github-actions[bot]]

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

[github-actions[bot]]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.