Currently tidb2dw need to connect to upstream database(TiDB) as well as send request to TiCDC to issue the incremental sinking. And tidb2dw need a lot of upstream context, this is not convenient when we deploy tidb2dw as a cloud service.
Proposal
Use S3 as the bridge between upstream databases and tidb2dw, tidb2dw doesn't connect to upstream components, tidb2dw just read the full backup and incremental data changes from S3.
Lloyd-Pottiger
commented
1 year ago
Background
Currently tidb2dw need to connect to upstream database(TiDB) as well as send request to TiCDC to issue the incremental sinking. And tidb2dw need a lot of upstream context, this is not convenient when we deploy tidb2dw as a cloud service.
Proposal