CohereAPIError thrown when base retriever returns empty documents in ContextualCompressionRetriever using Cohere Rank

hanguofeng commented 1 year ago

System Info

5.19.0-42-generic # 43~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 21 16:51:08 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
langchain==0.0.180
Python 3.10.11

Who can help?

No response

Information

[ ] The official example notebooks/scripts
[X] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[X] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction

Set up a retriever using any type of retriever (for example, I used Pinecone).
Pass it into the ContextualCompressionRetriever.
If the base retriever returns empty documents,
It throws an error: cohere.error.CohereAPIError: invalid request: list of documents must not be empty

File "/workspaces/example/.venv/lib/python3.10/site-packages/langchain/retrievers/contextual_compression.py", line 37, in get_relevant_documents compressed_docs = self.base_compressor.compress_documents(docs, query) File "/workspaces/example/.venv/lib/python3.10/site-packages/langchain/retrievers/document_compressors/cohere_rerank.py", line 57, in compress_documents results = self.client.rerank( File "/workspaces/example/.venv/lib/python3.10/site-packages/cohere/client.py", line 633, in rerank reranking = Reranking(self._request(cohere.RERANK_URL, json=json_body)) File "/workspaces/example/.venv/lib/python3.10/site-packages/cohere/client.py", line 692, in _request self._check_response(json_response, response.headers, response.status_code) File "/workspaces/example/.venv/lib/python3.10/site-packages/cohere/client.py", line 642, in _check_response raise CohereAPIError( cohere.error.CohereAPIError: invalid request: list of documents must not be empty

Code is Like


retriever = vectorstore.as_retriever()
compressor = CohereRerank()
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor, base_retriever=retriever
)
return compression_retriever

Expected behavior

no error throws and return empty list

dev2049 commented 1 year ago

good catch! happy to review if you want to open a fix, otherwise can get to it this weekend