ElasticsearchItemReader keeps grabbing data indefinitely because of the implementation of the doPageRead() method.

[theflofly]

The method doPageRead() from ElasticsearchItemReader will stop if the ES query return null.

So normally if there are 100 items to retrieve with a 50 items range, the method doPageRead() is called three times. The first time 50 items are retrieved, the second time 50 others and the last time, the query returns null so doPageRead() stops.

Here the query keeps retrieving indefinitely the 50 first items, even if the SearchQuery is paginated.

I will find a solution, then share it here.

[theflofly]

So here is the soluce so that the doPageRead() method uses a Pageable. If I don't use the following code the doPageRead() never ends.

@Override
@SuppressWarnings("unchecked")
protected Iterator<T> doPageRead() {

    logger.debug("executing query {}", query.getQuery());
    Iterator<T> itT = (Iterator<T>)elasticsearchOperations.queryForList(query, targetType).iterator();

    // the pageable can be null so Optional is useful here
    Optional<Pageable> optionalPageable = Optional.ofNullable(query.getPageable());
    query.setPageable(optionalPageable.get().next());

    return itT;

}

By default the SearchQuery has a Pageable from 0 to 10 items. You can change these settings when you create the SearchQuery using the NativeSearchQueryBuilder.