I ran kafka in docker image :
Tested that with sample python application. Able to send message and cosume messages.
created image of opentelemetry contrib with below configuration`receivers:
otlp:
protocols:
http:
endpoint: 0.0.0.0:4321
grpc:
endpoint: 0.0.0.0:4322
processors:
batch:
timeout: 1s
exporters:
kafka:
topic: testtopic
brokers: [kafkaserver:9092]
metadata:
full: false
retry:
max: 3
backoff: 1000ms
timeout: 10s
retry_on_failure:
enabled: true
initial_interval: 2s #Time to wait after the first failure before retrying; ignored if enabled is false. (default = 5s)
max_interval: 10s # (default = 30s): Is the upper bound on backoff; ignored if enabled is false
max_elapsed_time: 60s # (default = 120s): Is the maximum amount of time spent trying to send a batch; ignored if enabled is false
auth:
tls:
insecure: true
producer:
max_message_bytes: 10000000
required_acks: -1
protocol_version: 2.0.0
service:
extensions: [pprof, zpages, health_check]
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [kafka]`
Getting error: 2022-10-13T00:45:20.875Z info exporterhelper/queued_retry.go:215 Exporting failed. Will retry the request after interval. {"kind": "exporter", "data_type": "traces", "name": "kafka", "error": "### Failed to deliver 1 messages due to kafka: client has run out of available brokers to talk to: dial tcp 127.0.0.1:9092: connect: connection refused", "interval": "1.12512345s"}
I ran kafka in docker image :
Tested that with sample python application. Able to send message and cosume messages.
created image of opentelemetry contrib with below configuration`receivers:
otlp:
protocols:
http:
endpoint: 0.0.0.0:4321
grpc:
endpoint: 0.0.0.0:4322
processors: batch: timeout: 1s
exporters: kafka: topic: testtopic brokers: [kafkaserver:9092] metadata: full: false retry: max: 3 backoff: 1000ms timeout: 10s retry_on_failure: enabled: true initial_interval: 2s #Time to wait after the first failure before retrying; ignored if enabled is false. (default = 5s) max_interval: 10s # (default = 30s): Is the upper bound on backoff; ignored if enabled is false max_elapsed_time: 60s # (default = 120s): Is the maximum amount of time spent trying to send a batch; ignored if enabled is false auth: tls: insecure: true producer: max_message_bytes: 10000000 required_acks: -1 protocol_version: 2.0.0
service:
extensions: [pprof, zpages, health_check]
pipelines: traces: receivers: [otlp] processors: [batch] exporters: [kafka]` Getting error: 2022-10-13T00:45:20.875Z info exporterhelper/queued_retry.go:215 Exporting failed. Will retry the request after interval. {"kind": "exporter", "data_type": "traces", "name": "kafka", "error": "### Failed to deliver 1 messages due to kafka: client has run out of available brokers to talk to: dial tcp 127.0.0.1:9092: connect: connection refused", "interval": "1.12512345s"}
any suggestions?