DataCater / datacater

The developer-friendly ETL platform for transforming data in real-time. Based on Apache Kafka® and Kubernetes®.
https://datacater.io
Other
82 stars 4 forks source link

Native pipeline image does not pick up stream configs #94

Closed flippingbits closed 1 year ago

flippingbits commented 1 year ago

We provide the stream configs to pipelines as environment variables.

It looks like the native pipeline image does not pick up the stream configs but tries to use default values.

When I create a deployment with the native image, I see the following error messages in the logs (despite using a remote Kafka cluster on Confluent Cloud):

2022-12-16T13:23:20.455Zio.smallrye.healthSRHCK01001: Reporting health down status: {"status":"DOWN","checks":[{"name":"SmallRye Reactive Messaging - liveness check","status":"UP","data":{"stream-in":"[OK]","stream-out":"[OK]"}},{"name":"SmallRye Reactive Messaging - readiness check","status":"DOWN","data":{"stream-in":"[KO]","stream-out":"[KO]"}},{"name":"SmallRye Reactive Messaging - startup check","status":"DOWN","data":{"stream-in":"[KO]","stream-out":"[KO]"}}]}
2022-12-16T13:23:23.324Zorg.apache.kafka.clients.NetworkClient[Producer clientId=kafka-producer-stream-out] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
2022-12-16T13:23:23.325Zorg.apache.kafka.clients.NetworkClient[Producer clientId=kafka-producer-stream-out] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
flippingbits commented 1 year ago

Fixed by #99