search

bakdata / kpops

Deploy Kafka pipelines to Kubernetes
https://bakdata.github.io/kpops
MIT License
12 stars 1 forks source link

Handle topics with Strimzi Topic Operator #351

Open raminqaf opened 1 year ago

raminqaf commented 1 year ago

Currently, we are relying on Kafka Rest Proxy to create the input/output topics. An additional way of creating topics would be through Strimzi's topic operator. This operator handles the topics as Kubernetes resources. In this way, we can handle topics as pipeline components.

disrupted commented 1 year ago

👍🏻 I believe this depends on the refactoring I was talking about, making topics first-class citizens in KPOps first. Then, the deployment backend (be it Kafka REST, Kubernetes manifests, etc.) can be interchangeable/customizable.