search

castorini / rank_llm

Repository for prompt-decoding using LLMs (GPT3.5, GPT4, Vicuna, and Zephyr)
http://rankllm.ai
Apache License 2.0
277 stars 36 forks source link

P1-Retrieved Results Cache #33

Open ronakice opened 6 months ago

ronakice commented 6 months ago

Provide important cached retrieve results as well as rerank results hosted elsewhere but documented here. I can perhaps do this sometime.

sahel-sh commented 5 months ago

@ronakice it seems like you are looking for volunteers for this one, I unassigned it so that people can pick

ronakice commented 5 months ago

Updating this with the following link: https://github.com/castorini/rank_llm_data

ronakice commented 5 months ago

Please interface with this!

AndreSlavescu commented 5 months ago

interested

ronakice commented 5 months ago

Something like:

If it exists and matches md5 use it, else get it from rank_llm_data if there, else run retrieve.

Eventually, the rerank_results can be added to rank_llm_data too for verification. But not priority for now. This will likely fall into place after we have a nice 2CR after #32 is mature

ronakice commented 5 months ago

This is super important so bumping to P1. More people want to repro the baselines before jumping into their own dataset and making them download SPLADE indexes every time is probably not optimal