search

opensearch-project / opensearch-java

Java Client for OpenSearch
Apache License 2.0
123 stars 184 forks source link

[FEATURE] BulkRequest estimated size #638

Open andrewparmet opened 1 year ago

andrewparmet commented 1 year ago

Is your feature request related to a problem?

The RestHighLevelClient had a convenience method on BulkRequest that allowed us to dynamically batch bulk operations without exceeding OpenSearch's max request size.

What solution would you like?

Is this available in the current client? If not, can it be added? I don't mind porting the old implementation if needed.

What alternatives have you considered?

I'm implementing it myself but I have to use reflection to access private fields on UpdateOperationData such as the script to get its length if I don't want to serialize it just to get its size and then have the library reserialize it again when it sends it as part of the request.

Do you have any additional context?

Nope!

andrewparmet commented 1 year ago

I'm also curious about the implementation itself... why is script length doubled and update operations added without the request overhead?

peter-ponzel commented 3 weeks ago

are there any news on that topic ?

dblock commented 3 weeks ago

This needs to be investigated and implemented, we welcome your contribution!