DataLinkDC / dinky

Dinky is a real-time data development platform based on Apache Flink, enabling agile data development, deployment and operation.
http://www.dinky.org.cn
Apache License 2.0
3.15k stars 1.16k forks source link

[Bug] [CDCSOURCE] Oracle CDC could not deal with the date and number type properly #3478

Closed medivh511 closed 1 month ago

medivh511 commented 6 months ago

Search before asking

What happened

Flink 1.18.1 Dinky 1.02 Flink CDC 3.01

Oracle CDC could not transform the date and number type (especially for number(n) type), even u set the debezium.decimal.handling.mode to string

the exception looks like

number transform exception:

Caused by: java.lang.ClassCastException: class java.lang.Integer cannot be cast to class java.lang.String (java.lang.Integer and java.lang.String are in module java.base of loader 'bootstrap')

date transform exception:

Caused by: java.lang.ClassCastException: class java.lang.long cannot be cast to class java.lang.String (java.lang.Integer and java.lang.String are in module java.base of loader 'bootstrap')

What you expected to happen

The function initTypeConverterList in AbstractSinkBuilder did not work properly, not considered the oracle number and date type

How to reproduce

Create an oracle table which have date type and number(n) type's columns

run the script in dinky:

EXECUTE CDCSOURCE cdc_oracle WITH (  'connector'='oracle-cdc',  'hostname'='...',  'port'='1521',  'username'='...',  'password'='...',  'database-name'='...',  'checkpoint'='3000',  'scan.startup.mode'='initial',  'parallelism'='1',  'table-name'='....',  'debezium.log.mining.strategy'='online_catalog',  'debezium.database.history.store.only.captured.tables.ddl'='true',  'debezium.database.tablename.case.insensitive'='false', 'debezium.log.mining.continuous.mine'='true',  'debezium.decimal.handling.mode'='string',  'sink.connector'='upsert-kafka',  'sink.properties.bootstrap.servers'='....',  'sink.topic'='#{tableName}',  'sink.key.format'='json',  'sink.value.format'='json' )

Anything else

No response

Version

1.0.2

Are you willing to submit PR?

Code of Conduct

github-actions[bot] commented 6 months ago

Hello @medivh511, this issue is about CDC/CDCSOURCE, so I assign it to @aiwenmo. If you have any questions, you can comment and reply.

你好 @medivh511, 这个 issue 是关于 CDC/CDCSOURCE 的,所以我把它分配给了 @aiwenmo。如有任何问题,可以评论回复。

github-actions[bot] commented 4 months ago

Hello @, this issue has not been active for more than 30 days. This issue will be closed in 7 days if there is no response. If you have any questions, you can comment and reply.

你好 @, 这个 issue 30 天内没有活跃,7 天后将关闭,如需回复,可以评论回复。