searchWithScroll and searchWithScrollConfigurable functions were added:
the functions implement Elasticsearch scroll search logic (with searchWithScrollConfigurable accepting a step size as input param, for testing)
the data source will use a simple search if the requested size is <= maxSimpleSearchResultsSize (10000), and use scrolling search otherwise
2 integration tests added to verify that the data source makes a correct selection with the large size, and to verify that the multiple-search-steps logic works
What was changed
searchWithScroll
andsearchWithScrollConfigurable
functions were added:searchWithScrollConfigurable
accepting a step size as input param, for testing)maxSimpleSearchResultsSize
(10000
), and use scrolling search otherwiseFixes https://github.com/blackstork-io/fabric/issues/167