search

castorini / rank_llm

RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking.
http://rankllm.ai
Apache License 2.0
294 stars 38 forks source link

Adding Llama.cpp support with quantized models #16

Open ArthurCamara opened 11 months ago

ArthurCamara commented 11 months ago

Adding support to Llama.cpp with quantized models.

8-bit model: https://huggingface.co/castorini/rank_vicuna_7b_v1_q8_0/ 4-bit model: https://huggingface.co/castorini/rank_vicuna_7b_v1_q4_0/