search

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.85k stars 4.25k forks source link

In stream mode, ElasticsearchIO transform becomes unavailable #21053

Open damccorm opened 2 years ago

damccorm commented 2 years ago

If you use the ElasticSearchIO transform for a workflow in stream mode, and if something happens temporarily between the RestClient and the server, any future calls will result in a 

java.io.IOException: Connection reset by peer As there is not reconnection system ou keepalive enable, the client seems not able to reconnect and the workflow cannot work anymore.

Imported from Jira BEAM-12577. Original Jira may contain additional context. Reported by: jgallon.

RafaelAdao commented 11 months ago

Hello Apache Beam team,

I am facing the same issue described by jgallon in issue #21053, where using the ElasticSearchIO transformer in streaming mode results in java.io.IOException: Connection reset by peer after temporary failures between the RestClient and the server. I notice that this issue has been open for over a year with no activity.

I would like to inquire if there has been any progress in resolving this issue since then. I am experiencing the same situation in my environment and am willing to contribute in any way necessary to help address this issue.

Thank you for your attention.

Best regards, RafaelAdao