search

naver / bergen

Benchmarking library for RAG
Other
91 stars 8 forks source link

multi-gpu support for retrieval and re-ranking #8

Open davidmrau opened 3 months ago

davidmrau commented 3 months ago

Currently, multi-gpu support for retrieval and reranking is not or is partially available.

thibault-formal commented 3 months ago

hey! I think multi-gpu is already supported for retrieval?

davidmrau commented 3 months ago

When I tried only a single GPU was utilized, but we would have to check properly. Also, I have seen some DataParallel in Splade, which i think is not implemented in the dense one. So we should make sure we have a consistent setup and that it indeed utilizes all GPUs.

thibault-formal commented 3 months ago

I also added DataParallel in the dense class, so I think it should work (didn't try though)

thibault-formal commented 3 months ago

TODO @thibault-formal : check the gpu thing for retrieval

sclincha commented 3 weeks ago

@thibault-formal ?