Searches against a GPU index using normalize_L2 is heavily CPU intensive

[jpletcher]

Summary

Running searches against a GPU index using normalize_L2 is heavily CPU intensive.

Please refer to the following psuedo code:

def setup_index(numpy_array):
    res = faiss.StandardGpuResources()
    flat_index = faiss.IndexFlatIP(n)
    gpu_index = faiss.index_cpu_to_gpu(res, 0, flat_index)
    faiss.normalize_L2(numpy_array)
    gpu_index.add(numpy_array)
    return gpu_index

def query(query_vectors):    # 1024 32-bit floats
    faiss.normalize_L2(query_vectors)
    distances, indices = gpu_index.search(query_vector, VALUE_OF_K)

When invoking query(), we find that the platform is CPU constrained where CPU resources are consumed at fvec_renorm_L2 (faiss/swigfaiss.py), while GPU utilization remains stable.

• In this psuedo code above, can normalize_L2 be optimized to utilize GPU?

Additionally, for this case, AVX2 support is not available.

• Would AVX2 support provide any performance optimizations to normalize_L2 ?

Platform

OS: Linux

Faiss version: faiss-gpu-1.7.2

Installed from: pip

Faiss compilation options: N/A

Running on:

• [ ] CPU
• [X] GPU

Interface:

• [ ] C++
• [X] Python

Reproduction instructions

[junjieqi]

@jpletcher

1. The normalize_l2 is only implemented on CPU so we can't utilize GPU for normalize_2 method for now.

2. Right now, the normalzie_l2 implementation should leverage autovectorization https://github.com/facebookresearch/faiss/blob/34bbe5e540bbc4edd0de38cb98bf0a563b2bae45/faiss/utils/distances_simd.cpp#L200-L209