Multi-GPU Sharding

[gorlando04]

Summary

Platform

4 NVIDIA GeForce RTX 2080 Super Ubuntu 20.04 Cuda 12

Faiss version: 1.7.3

Installed from: anaconda

Faiss compilation options:

Running on:

• [ ] CPU
• [x ] GPU

Interface:

• [ ] C++
• [x ] Python

Reproduction instructions

I am trying to make a knn search using faiss in GPU using the multi-gpu indexes with sharding. But when im trying to execute the code it keeps to get MemoryError.

import numpy as np from time import time d = 12 # dimension nb = 40000000 # database size np.random.seed(1234) # make reproducible xb = np.random.random((nb, d)).astype('float32')

import faiss # make faiss available

Recall func

def recall(arr1,arr2,k):

#Verificação da integridade
if arr1.shape != arr2.shape:
    print("Impossível de fazer a avaliação, as arrays tem tamanho diferentes")
elif arr1.shape[1] < k:
    print(f"Impossível de fazer o recall{k}, já que as array não tem {k} vizinhos")

#Somatório dos k primeiros vizinhos positivos dividido por n*k
acertos = 0

n = arr1.shape[0]

recall_value = (arr1[:,:k] == arr2[:,:k]).sum() / (float(n*k))

return recall_value

ngpus = faiss.get_num_gpus()

print("number of GPUs:", ngpus)

for i in range(20000,1000,-512): k = 20 t0 = time() qntd = faiss.IndexFlatL2(d) cpu_index = faiss.IndexIVFFlat(qntd,d,i,faiss.METRIC_L2) cpu_index.nprobe = 13 print(f"Nlist = {i} nprobe = {cpu_index.nprobe}") print("Fazendo o indice") p = faiss.GpuMultipleClonerOptions() p.shard = True p.common_ivf_quantizer = True gpu_index = faiss.index_cpu_to_all_gpus( # build the index cpu_index,p ) print("Training") gpu_index.train(xb) gpu_index.add(xb) # add vectors to the index print(f"Tempo de treino = {time()-t0}")

                    # we want to see 4 nearest neighbors
print("Search")
_, I = gpu_index.search(xb, k) # actual search
print(f"Time = {time()-t0}")

[mdouze]

This is probably because you are calling faiss.index_cpu_to_all_gpus in a loop, this allocates a StandardGpuResources object that uses about 1.5G of GPU RAM.

[gorlando04]

What should I use instead?