search

toptal / chewy

High-level Elasticsearch Ruby framework based on the official elasticsearch-ruby client
MIT License
1.88k stars 366 forks source link

Support PITs (points in time) and search_after #872

Open abartov opened 1 year ago

abartov commented 1 year ago

Paging through large result sets (in my case, about 40k results) is memory expensive when just increase the max result window. The recommended method by Elastic is using PITs and search_after, but I don't see support for that in Chewy.

It would be great to be able to use search_after, or even to benefit from it when requesting it with a flag, and simply calling .page() to get a particular results page.

BagpipesJohnson commented 1 year ago

We also would like to be able to use point_in_time. If there is a way to do this without forking chewy ourselves, that would be huge.

https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html

abartov commented 1 year ago

Perhaps @konalegi or someone could at least review this feature request and indicate whether support is planned?

konalegi commented 1 year ago

Added to my todo list to take a look, I cannot give quick answer right now.

BagpipesJohnson commented 1 year ago

Is there any way that we can add to the request body ourselves?