Open meghbhalerao opened 11 months ago
meghbhalerao
commented
11 months ago Actually, I think I might have figured out how to do it, I can just chunk my queries into batches and then loop over those chunks and do index.search(chunk, kneighbors), which would go something like -
index.add(whole_data)
chunks = chunk_data(whole_data, num_chunks)
for chunk in chunks:
index.search(chunk, kneighbors)
mdouze
commented
11 months ago yes please do. We recently introduced batching for large GPU queries but it is not available everywhere.
gajghatenv
commented
7 months ago @meghbhalerao where did you find docs for chunking?
meghbhalerao
commented
7 months ago I don't think I used any docs. The code which I used above simply does the matrix matrix multiply in chunks rather than all at once.
gajghatenv
commented
7 months ago I am using the chunk of code you had in your response and I still get the error from your original post
meghbhalerao
commented
7 months ago You might have to reduce the chunk size, maybe? So that it fits in the memory of your GPU? or it might be just that your index might be too large?
On Wed, Jan 10, 2024 at 1:49 PM gajghatenv @.***> wrote:
I am using the chunk of code you had in your response and I still get the error from your original post
— Reply to this email directly, view it on GitHub https://github.com/facebookresearch/faiss/issues/3049#issuecomment-1885793845, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5KGEBCMPXIXCAQ4HMUXDDYN4EF7AVCNFSM6AAAAAA4SUJX5OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVG44TGOBUGU . You are receiving this because you were mentioned.Message ID: @.***>
-- Thanks & Regards, Megh Bhalerao B.Tech in Electrical & Electronics Engineering Homepage: https://meghbhalerao.github.io https://meghbhalerao.github.io
Summary
Hi, I want to do exact NN search for 11M samples and 512 features. Hence I have a feature matrix of 11M x 512
Platform
GPU specifications are the following, obtained via
nvidia-smicommand -Faiss version:
Installed from:
conda 23.1.0Faiss compilation options:
Running on: GPU Interface: Python
Reproduction instructions
Following is the minimum working example to reproduce the issue -
Code for minimum working example - this is just using a random matrix, but in the real codebase we of course use a matrix of features -
Summary
Hi, I want to do exact NN search for 11M samples and 512 features. Hence I have a feature matrix of 11M x 512
Platform
GPU specifications are the following, obtained via
nvidia-smicommand -Faiss version:
Installed from:
conda 23.1.0Faiss compilation options:
Running on: GPU Interface: Python
Reproduction instructions
Following is the minimum working example to reproduce the issue -
Code for minimum working example - this is just using a random matrix, but in the real codebase we of course use a matrix of features -
Logs -
Is there some way to do exact NN search, i.e. I still want to use the
IndexFlatL2, but say calculate the distances in batches so that it does not give an OOM error, i.e. if that is the cause of the error in the first place? I am a little unsure of the cause of the error.The data is of size 22GB and the GPU memory is 80GB, so I think that should not be a problem, since it fits in the RAM, so I am not sure what the problem is?
Please do let me know if I am missing anything and thank you very much for your time!