search

microsoft / LLMLingua

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
https://llmlingua.com/
MIT License
4.27k stars 228 forks source link

Why no integration with Langchain till now? #71

Closed AIAnytime closed 4 months ago

AIAnytime commented 6 months ago

I can see there is no docs available for community to use LLMLingua with Langchain. We have with Llama Index but not langchain. Is there any example how one can use this in Langchain for RAG use cases?

iofu728 commented 6 months ago

Hi @AIAnytime,

Currently, we do not have an official implementation of LLMLingua for Langchain. If you are willing to contribute your implementation, I would be delighted to feature it on our homepage. Details regarding the implementation can be found in #31. Thank you once again for your support.

iofu728 commented 4 months ago

Thanks to @thehapyone's contribution, LLMLingua is now available in Langchain. You can follow this notebook for guidance.

AIAnytime commented 4 months ago

Thanks to @thehapyone's contribution, LLMLingua is now available in Langchain. You can follow this notebook for guidance.

Which notebook?

couturierc commented 4 months ago

Which notebook?

https://github.com/langchain-ai/langchain/blob/master/docs/docs/integrations/retrievers/llmlingua.ipynb

AIAnytime commented 4 months ago

I can confirm this is not working with Langchain and it gives below error:

`--------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) in <cell line: 2>() 1 from langchain.retrievers import ContextualCompressionRetriever ----> 2 from langchain_community.retrievers.document_compressors import LLMLinguaCompressor 3 from langchain_openai import ChatOpenAI

ModuleNotFoundError: No module named 'langchain_community.retrievers.document_compressors'`

AIAnytime commented 4 months ago

Which notebook?

https://github.com/langchain-ai/langchain/blob/master/docs/docs/integrations/retrievers/llmlingua.ipynb

`--------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) in <cell line: 2>() 1 from langchain.retrievers import ContextualCompressionRetriever ----> 2 from langchain_community.retrievers.document_compressors import LLMLinguaCompressor 3 from langchain_openai import ChatOpenAI

ModuleNotFoundError: No module named 'langchain_community.retrievers.document_compressors'`

I double checked that I have installed it but it gives the error.

thehapyone commented 4 months ago

You have to wait for the langchain team to release a new version of the community package - https://github.com/langchain-ai/langchain/releases

mmilannaik commented 3 months ago

You have to wait for the langchain team to release a new version of the community package - https://github.com/langchain-ai/langchain/releases

Is the issue got fixed for langchain? I can see there is a new release of langchain two days ago