pimcore / generic-data-index-bundle

Other
3 stars 1 forks source link

Pagination for big amounts of data #129

Closed markus-moser closed 2 months ago

markus-moser commented 3 months ago

In the studio UI backend the pagination will look similar to the current classic UI:

image

OpenSearch by default has a limit of 10.000 for the offset of a search result. So pagination for pages after this limit will fail (unless not changed in the index configuration).

Therefore we need to somehow tackle this problem. I think the best solution might be to somehow inverse the sort direction for the pages at the end - so when jumping to the last page of a search result in ascending order would result on the first page in descending order. With this approach and the default limit it would be possible to page over 20.000 items. The result array would be needed to be flipped afterwards.

With this approach we would only get a problem for pages in the middle. For those pages we could use the scroll API, but unfortunately it's not possible to directly jump to a page with it (and also the performance would be a real problem). So there is no real solution for this case. I think the best approach would be to really show a error message to the user in this case and maybe play arround a little bit with the max_result_window setting if it is safe to increase it a little bit.