issues
search
microsoft
/
LLMLingua
To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
https://llmlingua.com/
MIT License
4.42k
stars
241
forks
source link
Feature(LongLLMLingua): support reranker model
#13
Closed
iofu728
closed
10 months ago
iofu728
commented
10 months ago
Add newly embedding models, like
bge
,
voyageai
,
jinza
Add newly reranked models, like
bge_reranker
,
bge_llmembedder
,
cohere
Add the script retrieval example in
https://github.com/microsoft/LLMLingua/blob/main/examples/Retrieval.ipynb