Closed kxleee closed 11 months ago
Hey @kxleee can you provide some additional information here to assist? For example are you using the async or sync client? Do you have some sample code you can share?
@kxleee Do you have an example please?
The helpers such as streaming_bulk
do support yielding data, e.g. https://github.com/opensearch-project/opensearch-py/blob/main/samples/bulk/bulk-helpers.py#L84
def _generate_data() -> Any:
for i in range(100):
yield {"_index": index_name, "_id": i, "value": i}
succeeded = []
failed = []
for success, item in helpers.streaming_bulk(client, actions=_generate_data()):
if success:
succeeded.append(item)
else:
failed.append(item)
I am going to close this since we didn't hear from @kxleee with an example. Let's reopen if they mean something else.
The bulk method cannot be used in combination with yield when batch inserts