facebookresearch / faiss

A library for efficient similarity search and clustering of dense vectors.
https://faiss.ai
MIT License
30.78k stars 3.59k forks source link

Clustering on a selected GPU #2763

Open dzenilee opened 1 year ago

dzenilee commented 1 year ago

Summary

Currently, this is what it says in the README:

Clustering on one or several GPUs can be done via the gpu=True (use all gpus) or gpu=3 (use 3 gpus) constructor option in the KMeans object.

So we can do something like:

kmeans = faiss.Kmeans(d, k, niter=niter, verbose=verbose, gpu=True)

But can we have the option to choose WHICH gpu(s) to use instead of specifying the number of gpus to use (or to use all gpus)?

faiss.index_cpu_to_gpu is only for a Faiss index, right?

Platform

OS:

Faiss version:

Installed from:

Faiss compilation options:

Running on:

Interface:

Reproduction instructions

mdouze commented 1 year ago

This precise configuration is not supported. You can say how many gpus you want with gpu=3 though. Refer to https://github.com/facebookresearch/faiss/blob/main/faiss/python/extra_wrappers.py#L456

matrixji commented 1 year ago

It seems could configure the CUDA_VISIBLE_DEVICES environment variable at CUDA level, this will configure CUDA only to see those devices.