Open Hargo-W opened 1 week ago
The opensearch output will not throw an error if it cannot reach opensearch. The consumed message will be acked.
running this config in a docker container using the latest RPC image:
input: generate: count: 1 mapping: 'root = "Error please"' output: opensearch: urls: ["foo"] index: "bar" action: "index" id: "123" logger: level: TRACE
I get these logs:
2024-11-14 14:56:06 benthos | level=info msg="Running main config from specified file" @service=redpanda-connect benthos_version=v4.33.0 path=/benthos.yaml 2024-11-14 14:56:06 benthos | level=info msg="Listening for HTTP requests at: http://0.0.0.0:4195" @service=redpanda-connect 2024-11-14 14:56:06 benthos | level=info msg="Input type generate is now active" @service=redpanda-connect label="" path=root.input 2024-11-14 14:56:06 benthos | level=trace msg="Consumed 1 messages from 'generate'." @service=redpanda-connect label="" path=root.input 2024-11-14 14:56:06 benthos | level=info msg="Launching a Redpanda Connect instance, use CTRL+C to close" @service=redpanda-connect 2024-11-14 14:56:06 benthos | level=info msg="Output type opensearch is now active" @service=redpanda-connect label="" path=root.output 2024-11-14 14:56:06 benthos | level=trace msg="Attempting to write 1 messages to 'opensearch'." @service=redpanda-connect label="" path=root.output 2024-11-14 14:56:06 benthos | level=debug msg="Successfully dispatched [%!s(uint64=0)] documents in 7ms (%!s(int64=0) docs/sec)" @service=redpanda-connect label="" path=root.output 2024-11-14 14:56:06 benthos | level=trace msg="Successfully wrote 1 messages to 'opensearch'." @service=redpanda-connect label="" path=root.output 2024-11-14 14:56:06 benthos | level=debug msg="Waiting for pending acks to resolve before shutting down." @service=redpanda-connect label="" path=root.input 2024-11-14 14:56:06 benthos | level=debug msg="Pending acks resolved." @service=redpanda-connect label="" path=root.input 2024-11-14 14:56:06 benthos | level=info msg="Pipeline has terminated. Shutting down the service" @service=redpanda-connect 2036-01-01 00:00:00 benthos exited with code 0
These logs state that the message was written successfully, even though there is no opensearch cluster to connect to.
I tested this setup with a kafka output as well as a redis output instead. Both of them throw errors and retry as expected.
Thank you!
The opensearch output will not throw an error if it cannot reach opensearch. The consumed message will be acked.
running this config in a docker container using the latest RPC image:
I get these logs:
These logs state that the message was written successfully, even though there is no opensearch cluster to connect to.
I tested this setup with a kafka output as well as a redis output instead. Both of them throw errors and retry as expected.
Thank you!