search

Altinity / clickhouse-sink-connector

Replicate data from MySQL, Postgres and MongoDB to ClickHouse®
https://www.altinity.com
Apache License 2.0
234 stars 54 forks source link

DDL not using target database name when enable.snapshot.ddl is set to true #863

Open MrDingz opened 1 month ago

MrDingz commented 1 month ago

Bug Description When using the clickhouse.database.override.map configuration, the connector should map the source database name to a new target database name. However, when enable.snapshot.ddl is set to "true", the connector parses and executes the DDL statements from the initial snapshot using the source database name, rather than the mapped target database name. As a result, the DDL statements are executed on the source database instead of the intended target database.

Steps to Reproduce

  1. Configure clickhouse.database.override.map to map the source database name to a new target database name.
  2. Set enable.snapshot.ddl to "true" to enable DDL parsing and execution during the initial snapshot.
  3. Observe that the DDL statements are executed using the source database name rather than the target database name specified in clickhouse.database.override.map.

Expected Behavior The DDL statements should use the target database name specified in clickhouse.database.override.map, ensuring that all DDL operations are executed in the target database.

Actual Behavior The DDL statements are executed using the source database name, which leads to incorrect execution in the wrong database.

Relevant Code The issue seems to be related to the DDL execution logic in DebeziumChangeEventCapture.java, line 143. The logic does not correctly apply the mapped target database name during DDL execution.

Environment

Additional Context This issue prevents the connector from correctly executing DDL in the desired target database when overriding database names, which limits the usefulness of the clickhouse.database.override.map configuration.

Please let me know if you need any further information or if I can assist with testing a potential fix.

subkanthi commented 2 weeks ago

@MrDingz we are not able to reproduce this, can you please provide an example of a specific DDL that failed