#1826 - Pagination and sorting for ElasticStream

[uladkaminski]

Generic request

• [x] PR name follows the pattern #1234 – issue name
• [x] branch name does not contain '#'
• [x] base branch (master or release/xx) is correct
• [x] PR is linked with the issue
• [x] task status changed to "Code review"
• [x] code follows product standards
• [x] regression tests updated

Implementation is done for the expected behaviour. Four cases were considered:

• The stream has sorting and limit: return all records. the amout limited by the result hits count or limit parameter.

  

• The steam has sorting only: return all records sorted. (even more then 10k)

  

• The steam has limit only: return the amount of records from the limit parameters but not more than 10k records.

  

• The stream has no limit and sorting: return not more then 10k records

  

[uladkaminski]

Several things I would probably need to clarify:

• Since we don't actually use Comparator, I introduced IndigoComparator structure to hold field and sort order for comparison.

• The search_after approach requires distinct value in the search result. That is why I am adding additional _doc sorting in case of any sorting was added.

• Current sorted method has accumulative effect. So every time the method is invoked additional sorting is added. Maybe it makes sense to reinitiate the sorting every time and introduce some kind of multiple field sorting?

• Except sorted(Comparator) Stream API contains default sorted() method which also can have some kind of default sorting logic. Do you have an idea what the logic can be?

• Not sure if this is the issue. Currently sorting by name is not allowed due to name is not a keyword. Let me know if this functionality is expected.