Is it possible to correlate records that were sent using batching mode with the array items (req []elastic.BulkableRequest or response *elastic.BulkResponse) in the "after" method? E.g. attach some metadata from the record when sending to the index, and retrieve that metadata in the "after" method, to post-process only the particular failed records.
We are using
[ ] elastic.v7 (for Elasticsearch 7.x)
Please describe the expected behavior
We expect to receive some metadata we can use to post-process the failed records. For example, we had a dead node and we are willing to send data to DLQ. To do so we need to have some clues on which of the records from the particular batch were failing and correlate them with the records that were put in a batch
Please describe the actual behavior
At the first glance, there are no specific values I can use to do the correlation neither from elastic.BulkableRequest nor elastic.BulkResponse, and I can only stop the process, return an error or do some other operations not related to particular unindexed records.
Is it possible to correlate records that were sent using batching mode with the array items (req []elastic.BulkableRequest or response *elastic.BulkResponse) in the "after" method? E.g. attach some metadata from the record when sending to the index, and retrieve that metadata in the "after" method, to post-process only the particular failed records.
We are using [ ] elastic.v7 (for Elasticsearch 7.x)
Please describe the expected behavior
We expect to receive some metadata we can use to post-process the failed records. For example, we had a dead node and we are willing to send data to DLQ. To do so we need to have some clues on which of the records from the particular batch were failing and correlate them with the records that were put in a batch
Please describe the actual behavior
At the first glance, there are no specific values I can use to do the correlation neither from elastic.BulkableRequest nor elastic.BulkResponse, and I can only stop the process, return an error or do some other operations not related to particular unindexed records.
Any steps to reproduce the behavior?
NA