apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.93k stars 1.79k forks source link

[Discuss][CDC] Support processing DDL change events #4672

Closed hailin0 closed 3 months ago

hailin0 commented 1 year ago

Search before asking

Description

SeaTunnel as CDC data synchronization, I think we need to support DDL change as a feature, I'm writing a draft design. and I want to hear from you all how you think it can be implemented in SeaTunnel.

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

hailin0 commented 1 year ago

https://github.com/apache/incubator-seatunnel/issues/4701

hailin0 commented 1 year ago

DDL SQL Parser:

Antlr supported dialects: https://github.com/antlr/grammars-v4/tree/master/sql

Calcite supported dialects: https://github.com/apache/calcite/tree/main/core/src/main/java/org/apache/calcite/sql/dialect

Adapte debezium ddl parser: https://github.com/debezium/debezium/tree/1.6/debezium-ddl-parser/src/main/antlr4/io/debezium/ddl/parser

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 1 year ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

hailin0 commented 1 year ago

Zeta engine adapter. https://github.com/apache/seatunnel/pull/5125

hailin0 commented 4 months ago

mysql-cdc connector adapter #6929