DataCater / datacater

The developer-friendly ETL platform for transforming data in real-time. Based on Apache Kafka® and Kubernetes®.
https://datacater.io
Other
82 stars 4 forks source link

Provide pipeline spec of deployments #50

Closed flippingbits closed 1 year ago

flippingbits commented 1 year ago

Deployments are associated with a specific pipeline uuid. Once a deployment has been created/started, users cannot retrieve the pipeline version/spec that the deployment is based on.

This might lead to confusion and unhappiness among users.

We store the pipeline spec in a Kubernetes Config Map that gets assigned to the pipeline Pod.

We should make the content of the Config Map, i.e., the pipeline spec, available in the Deployments endpoints of our API.