Open harshalschaudhari opened 1 month ago
Pinging code owners:
exporter/kafka: @pavolloffay @MovieStoreGuy
See Adding Labels via Comments if you do not have permissions to add labels yourself.
I believe this is related to #24029 TL;DR if Kafka is not available on startup the collector crashes
Note: I have tried out kafka using docker with port 29092
Pod Logs as below
2024-07-18 16:25:51 Error: cannot start pipelines: kafka: client has run out of available brokers to talk to: dial tcp 172.20.0.9:29092: connect: connection refused 2024-07-18 16:25:51 2024/07/18 10:55:51 collector server run finished with error: cannot start pipelines: kafka: client has run out of available brokers to talk to: dial tcp 172.20.0.9:29092: connect: connection refused
Additional information
I am able to connect Kafka using Offset Explorer 3.0.
OTEL Configuration
exporters:
kafka:
brokers:
- kafka:9092
any update?
Hey, Came to make an issue but my issue is near exact same.
Currently using alloy which leverages this component as a wrapper. Exact same error.
Fails with this component when reaching confluent cloud 9092 over a azure privatelinkservice connection (we have kafka connectors in the same subnet operating with no problems.)
Would be really nice to get this working as my organisation has some very unique DMZ requirements for a new project that this would solve perfectly.
Component(s)
exporter/kafka
What happened?
Description
The application generates traces that can be viewed in Jaeger. I attempted to configure the Otel collector to send these traces to Kafka using various combinations within a Docker Compose setup, but encountered the following error:
Steps to Reproduce
Expected Result
Kafka shows the traces in respective topic.
Actual Result
2024-07-17 16:56:13 2024-07-17T11:26:13.794Z info service@v0.104.0/service.go:115 Setting up own telemetry... 2024-07-17 16:56:13 2024-07-17T11:26:13.799Z info service@v0.104.0/telemetry.go:96 Serving metrics {"address": ":8888", "level": "Normal"} 2024-07-17 16:56:13 2024-07-17T11:26:13.816Z info service@v0.104.0/service.go:193 Starting otelcol... {"Version": "0.104.0", "NumCPU": 2} 2024-07-17 16:56:13 2024-07-17T11:26:13.820Z info extensions/extensions.go:34 Starting extensions... 2024-07-17 16:56:14 2024-07-17T11:26:14.592Z info service@v0.104.0/service.go:256 Starting shutdown... 2024-07-17 16:56:14 2024-07-17T11:26:14.592Z info extensions/extensions.go:59 Stopping extensions... 2024-07-17 16:56:14 2024-07-17T11:26:14.592Z info service@v0.104.0/service.go:270 Shutdown complete. 2024-07-17 16:56:14 Error: cannot start pipelines: kafka: client has run out of available brokers to talk to: dial tcp [::1]:9092: connect: connection refused 2024-07-17 16:56:14 2024/07/17 11:26:14 collector server run finished with error: cannot start pipelines: kafka: client has run out of available brokers to talk to: dial tcp [::1]:9092: connect: connection refused
Collector version
0.104.0
Environment information
Environment
Docker-Compose
otel-collector: image: otel/opentelemetry-collector:0.104.0 #otel/opentelemetry-collector:latest container_name: otel-collector volumes: - ./otel-config.yaml:/etc/otel/config.yaml ports: - "127.0.0.1:4317:4317" - "127.0.0.1:55679:55679" command: ["--config", "/etc/otel/config.yaml"] networks: - otel-network depends_on: - kafka
OpenTelemetry Collector configuration
Log output
No response
Additional context
No response