From a product perspective, currently TiCDC don't have the ability to verify the data correctness without stopping the replication process. From an engineer 's perspective, even after failing the validation check in the test process, it is still hard to debug the root cause because of a lack of information.
Describe the feature you'd like
Provides end to end verification without stopping the replication process, meanwhile providing module level check inside TiCDC, hope could provide more information and narrow down the root cause to module level if data correctness check fails.
Task breakdown
total 16 weeks, development & self-test 12 week, scenario test 4 week.
start with 1. e2e verification / TiDB as downstream (1.5 weeks) first,
then 2 (3 weeks).
finally 3 (1.5 weeks).
4 (2 weeks).
After finish continue with 1. e2e verification / MYSQL as downstream (2 weeks),
then 1. e2e verification / Kafka as downstream (4 weeks).
ben1009
commented
2 years ago
Is your feature request related to a problem?
From a product perspective, currently TiCDC don't have the ability to verify the data correctness without stopping the replication process. From an engineer 's perspective, even after failing the validation check in the test process, it is still hard to debug the root cause because of a lack of information.
Describe the feature you'd like
Provides end to end verification without stopping the replication process, meanwhile providing module level check inside TiCDC, hope could provide more information and narrow down the root cause to module level if data correctness check fails.
Task breakdown
total 16 weeks, development & self-test 12 week, scenario test 4 week.
Detail subtasks as follows