deepset-ai / haystack

:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
https://haystack.deepset.ai
Apache License 2.0
14.53k stars 1.71k forks source link

Adding a CohereRanker node using the Cohere Reranking endpoint #5148

Closed sjrl closed 1 year ago

sjrl commented 1 year ago

Is your feature request related to a problem? Please describe. On the Sol-Eng team we have found the SentenceTransformersRanker node to be extremely useful in improving retrieval results for document retrieval. This helps to improve pipelines such as Document Search, Retrieval Augmented Generation, and Retriever-Reader pipelines.

We would like to the capabilities of ranker nodes in Haystack by adding support for the Cohere reranking endpoint. Cohere provides a reranking endpoint (docs here) that we would like to use in our work. It includes both an English model and a Multi-lingual model. We are particularly interested in the multilingual model since there are not really many (or any) good open-source models that can be used for multiple languages.

Describe alternatives you've considered Continue to use only the Sentence Transformer Ranker node. We could try and circumvent the multi-language support by training our own models.

Additional context We are finding for Retrieval Augmented Question Answering that the retrieval step is extremely important (even more so than for extractive QA). This is because most of the generative models we use (e.g. OpenAI) perform best when only being provided a small top_k from the retriever node (e.g. 3-5 documents). This is even the case with models that have a long context window. We think expanding support for Ranker nodes would help us greatly increase the top_k retrieval results.

sjrl commented 1 year ago

I would be happy to provide the PR for this feature.

sjrl commented 1 year ago

I've added a draft PR of the CohereRanker node that has worked when I tested it in Google Colab.