[X] I searched in the issues and found nothing similar.
Motivation
In this case, Kafka-CDC is not suitable for tables with large data volumes.
Because there are many tables during the synchronization of the whole database, and the source table may be modified, we do not want to create the target table in advance.
We want Kafka-CDC to support specifying the multi-level partitioning fields and the number of buckets for each table in the parameters:
eg：
--table_conf partitions.tableName1.fields=col1,col2,col3
--table_conf partitions.tableName2.fields=col1,col2,col3
--table_conf bucketKey.tableName1.fields=col1,col2,col3
--table_conf bucketKey.tableName2.fields=col1,col2,col3
--table_conf bucketNum.tableName1=4
--table_conf bucketNum.tableName2=6
Search before asking
Motivation
In this case, Kafka-CDC is not suitable for tables with large data volumes. Because there are many tables during the synchronization of the whole database, and the source table may be modified, we do not want to create the target table in advance. We want Kafka-CDC to support specifying the multi-level partitioning fields and the number of buckets for each table in the parameters: eg： --table_conf partitions.tableName1.fields=col1,col2,col3 --table_conf partitions.tableName2.fields=col1,col2,col3 --table_conf bucketKey.tableName1.fields=col1,col2,col3 --table_conf bucketKey.tableName2.fields=col1,col2,col3 --table_conf bucketNum.tableName1=4 --table_conf bucketNum.tableName2=6
Solution
No response
Anything else?
No response
Are you willing to submit a PR?