search

deepset-ai / haystack

:mag: AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
https://haystack.deepset.ai
Apache License 2.0
17.64k stars 1.91k forks source link

Pagination for Search results #586

Closed kadzaki closed 3 years ago

kadzaki commented 3 years ago

Question What is the best way to add paging to search results?

tanaysoni commented 3 years ago

Hi @kadzaki, the reader models make it non-trivial to implement paging as we can't set offsets for results in the Finder class. So, for each request to the next page, the Finder's reader model will re-compute all result pages(and discard the previous ones), making the paging inefficient.

In the next weeks, we plan to improve the confidence scores/probabilities for answers in Haystack. With the improved scores, a possible approach could be to set a large default page size and filter out answers below a threshold score.

kadzaki commented 3 years ago

ok thanks!

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 21 days if no further activity occurs.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 21 days if no further activity occurs.

ugm2 commented 2 years ago

Hi! 😃 Is this still impossible to implement? What if we are not using readers? Thanks!

Olivier-Gabriel commented 9 months ago

Hi!

Same question here: no reader needed, I just want to paginate results from an EmbeddingRetriever.

Thanks!

touhi99 commented 2 months ago

Is pagination still a possible thing specially in the case of RAG? anyone has tried it?