Closed whm233 closed 6 months ago
Hi @whm233, thank you for your support and interest in LLMLingua. Although I'm not an expert in LangChain, based on my experience, I believe its usage in LangChain should be similar to that in LlamaIndex, i.e., operating at the Postprocessor-level or reranker-level.
I briefly reviewed the LangChain pipeline and think you'll need to extend the BaseDocumentCompressor
to implement a class similar to CohereRerank
, as seen here: https://github.com/langchain-ai/langchain/tree/master/libs/langchain/langchain/retrievers/document_compressors/cohere_rerank.py.
Afterward, use ContextualCompressionRetriever()
to replace the compression_retriever.
Thank you again for your interest.
Thanks to @thehapyone's contribution, LLMLingua is now available in Langchain. You can follow this notebook for guidance.
code like this:
prompt = PromptTemplate(template=prompt_template, input_variables=["context", "question"]) kc = RetrievalQA.from_llm(llm=qwllm, retriever=compression_retriever, prompt=prompt)