search

elastic / elasticsearch-java

Official Elasticsearch Java Client
Apache License 2.0
397 stars 228 forks source link

Return a CompletableFuture from BulkIngester.flush method #806

Open aidin36 opened 2 months ago

aidin36 commented 2 months ago

The Issue

In one case in our production app, we want to flush the bulk operations and make them immediately available for search. So, we do a flush on our BulkIngester instance, then we do a client.indices().refresh(). However, because the flush operation is asynchronous, most of the time before all the operations flush, the refresh gets called. So, some of the changes aren't immediately available for search.

The Solution

I returned a CompletableFuture from the flush method, so the user can wait for the flush to finish if required.

Documents

Sorry, I couldn't find where's the document for the flush method so I could update it. If you point me to it, I can update the return type there.

gradlew check

There are nine failing tests. But they were failing before my changes too.

cla-checker-service[bot] commented 2 months ago

💚 CLA has been signed

l-trotta commented 1 month ago

Hello! Before changing the code, have you tried the refresh option in the globalSettings of the BulkIngester? Setting it to true would refresh after every BulkRequest , making the new data available to search.

aidin36 commented 3 weeks ago

Hi (: Thanks for the suggestion. Ideally, I don't want to refresh on all the bulk requests and let Elastic Search decide when is the right time to do so. We only need it in a specific case.

Besides my use case, I believe it's a good idea to return a Future from all the async methods and allow the client to know what happened to the result of the call.