search

castorini / rank_llm

RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking.
http://rankllm.ai
Apache License 2.0
312 stars 39 forks source link

Sliding Window logic fix and additional token length in larger candidates #51

Closed ronakice closed 8 months ago

ronakice commented 8 months ago

I was trying to stress test and our sliding window logic seemed to be failing in irregular windows that might not align perfectly with the candidate set size (which is a majority of cases, besides our settings like (20, 10) on candidates of 20/100, etc.).

This should fix it.

Additionally, some non-breaking changes (for our (20, 10) experiments) when we are dealing with smaller windows (you don't need to fix a 300 token limit).

sahel-sh commented 8 months ago

LGTM,! given that my last comment is not a blocker Thank you!