castorini / rank_llm

RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking.

http://rankllm.ai

Apache License 2.0

312 stars 39 forks source link

P2-Batch Inference #34

Open ronakice opened 8 months ago

ronakice commented 8 months ago

Currently, we only support inference with a (single query, single subset of documents), but technically we could batch over the query dimension pretty easily, doing it over document subsets is not doable given that the next call is not inferable in the sequential process that is sliding windows.

AndreSlavescu commented 8 months ago

interested

ronakice commented 7 months ago

Bumping priority

AndreSlavescu commented 6 months ago

batch inf:

castorini / rank_llm

P2-Batch Inference #34

102