DataLinkDC / dinky

Dinky is a real-time data development platform based on Apache Flink, enabling agile data development, deployment and operation.
http://www.dinky.org.cn
Apache License 2.0
2.92k stars 1.07k forks source link

[Bug] [CDC SOURCE] datastream-doris-schema-evolution sync datetime error #3593

Open fanshu19 opened 2 weeks ago

fanshu19 commented 2 weeks ago

Search before asking

What happened

environment flink 1.17.0 dinky 1.0.1 flink cdc 2.4.2

EXECUTE CDCSOURCE demo_mssql_doris WITH ( 'connector' = 'mysql-cdc', 'hostname' = 'xxxxx', 'port' = '3306', 'username' = 'xxxxxx', 'password' = '123456', 'checkpoint' = '30000', 'scan.startup.mode' = 'initial', 'parallelism' = '1', 'table-name' = 'glocal.tp_users2', 'sink.connector' = 'datastream-doris-schema-evolution', 'sink.fenodes' = 'xxxxxx', 'sink.username' = '${doris_writer_username}', 'sink.password' = '${doris_writer_password}', 'sink.sink.enable-delete' = 'true', 'sink.sink.max-retries' = '1', -- 'sink.sink.batch.interval' = '60000', 'sink.sink.db' = 'temp', 'sink.sink.properties.format' ='json', 'sink.sink.properties.read_json_byline' ='true', 'sink.table.prefix' = 'xxxx', 'sink.table.lower' = 'true', 'sink.table.identifier' = '#{schemaName}.#{tableName}', -- 字段跟随变更 'sink.sink.use-new-schema-change'='true' -- 'sink.sink.label-prefix' = '#{schemaName}#{tableName}_datastream_1' );

我的表里有datetime数据类型,但是用datastream-doris-schema-evolution插入到doris数据库的时候,datetime变成了时间戳,导致入库的时候变成了null。而如果用doris形式插入,又正常的。 我看代码有写过类似的转换,但是没有引用。可以完善一下这部分功能吗?

What you expected to happen

None

How to reproduce

None

Anything else

No response

Version

1.0.1

Are you willing to submit PR?

Code of Conduct

github-actions[bot] commented 2 weeks ago

Hello @fanshu19, this issue is about CDC/CDCSOURCE, so I assign it to @aiwenmo. If you have any questions, you can comment and reply.

你好 @fanshu19, 这个 issue 是关于 CDC/CDCSOURCE 的,所以我把它分配给了 @aiwenmo。如有任何问题,可以评论回复。