Right now the tidb2dw is more like a standalone tool that can connect stream data in blob storage to big data cloud platform like snowflake and redshift. When it comes to TiDB Cloud, when we want to connect a tenant's data to big cloud platform, there would be a dedicated tidb2dw component to sync the data changes. This makes the deployment and maintenance more complicate.
Proposal
Evolute tidb2dw as a service when deploy on the tidb cloud, so we don't need to deploy a new tidb2dw component when there is a new sync to big data cloud platform.
Background
Right now the tidb2dw is more like a standalone tool that can connect stream data in blob storage to big data cloud platform like snowflake and redshift. When it comes to TiDB Cloud, when we want to connect a tenant's data to big cloud platform, there would be a dedicated tidb2dw component to sync the data changes. This makes the deployment and maintenance more complicate.
Proposal
Evolute tidb2dw as a service when deploy on the tidb cloud, so we don't need to deploy a new tidb2dw component when there is a new sync to big data cloud platform.
Task breakdowns