search

pingcap / tidb-operator

TiDB operator creates and manages TiDB clusters running in Kubernetes.
https://docs.pingcap.com/tidb-in-kubernetes/
Apache License 2.0
1.22k stars 489 forks source link

Manage TiCDC replication tasks in CRD #2873

Open kolbe opened 4 years ago

kolbe commented 4 years ago

Feature Request

Is your feature request related to a problem? Please describe:

Today, after deploying ticdc using tidb-operator it's necessary to manually manage replication tasks using the cdc CLI. This doesn't fit very well with the approach used by Kubernetes users to manage their resources.

Describe the feature you'd like:

It would be nice to be able to define a TiCDC replication task in a CRD and have tidb-operator manage the task.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

DanielZhangQD commented 4 years ago

Is it possible to manage the task with REST API? @weekface Could you please help check this issue?

kolbe commented 4 years ago

It's certainly possible to use the CDC's REST API to directly manage these tasks, but it requires a customer to write an additional API client instead of being able to simply interact with the K8s API.

DanielZhangQD commented 4 years ago

Yes, good point.

kolbe commented 4 years ago

I think we could set a goal to define a few pretty common use-cases and encapsulate/manage those in a CRD. Replication from a TiDB source to an arbitrary MySQL-compatible replica, obviously, but maybe also replication from a TiDB source to another TiDB cluster managed within the same k8s cluster, or to a specific set of non-TiDB sinks.

weekface commented 4 years ago

Is it possible to manage the task with REST API? @weekface Could you please help check this issue?

CDC does not currently have the REST API of managing replication tasks.

DanielZhangQD commented 4 years ago

@weekface Please help evaluate if we can support the CDC cli with CRD + controller for the task management.