DataLinkDC / dinky

Dinky is a real-time data development platform based on Apache Flink, enabling agile data development, deployment and operation.
http://www.dinky.org.cn
Apache License 2.0
3.06k stars 1.13k forks source link

[Bug] [CDCSOURCE] When using mysql cdc to print, a ClassNotFound error is prompted #3779

Open diaozhonghao opened 1 week ago

diaozhonghao commented 1 week ago

Search before asking

What happened

dinky 1.0.0 flink 1.17 部署模式是Application

执行该语句时发生了错误 Caused by: java.lang.ClassNotFoundException: org.apache.doris.flink.sink.writer.serializer.DorisRecordSerializer image

只是打印,就提示doris依赖缺失

What you expected to happen

正常我没用到doris,不应该出现doris的需求 能够提供对应依赖或者是解决对应问题

How to reproduce

dinky 1.0.0 flink 1.17

extends/flink1.17/
├── antlr-runtime-3.5.2.jar
├── commons-cli-1.7.0.jar
├── dinky
│   ├── dinky-app-1.17-1.0.0-jar-with-dependencies.jar
│   ├── dinky-catalog-mysql-1.17-1.0.0.jar
│   └── dinky-client-1.17-1.0.0.jar
├── dinky-app-1.17-1.0.0-jar-with-dependencies.jar
├── druid-1.2.21.jar
├── flink-cep-1.17.2.jar
├── flink-connector-files-1.17.2.jar
├── flink-connector-jdbc-3.1.2-1.17.jar
├── flink-csv-1.17.2.jar
├── flink-dist-1.17.2.jar
├── flink-json-1.17.2.jar
├── flink-scala_2.12-1.17.2.jar
├── flink-shaded-guava-31.1-jre-17.0.jar
├── flink-shaded-hadoop-3-uber-3.1.1.7.2.9.0-173-9.0.jar
├── flink-sql-connector-hive-3.1.3_2.12-1.17.2.jar
├── flink-sql-connector-kafka-1.17.2.jar
├── flink-sql-connector-mysql-cdc-3.0.1.jar
├── flink-table-api-java-bridge-1.17.2.jar
├── flink-table-api-java-uber-1.17.2.jar
├── flink-table-planner_2.12-1.17.2.jar
├── flink-table-runtime-1.17.2.jar
├── hadoop-yarn-server-timelineservice-3.2.2.jar
├── HikariCP-5.0.1.jar
├── hive-exec-3.1.3.jar
├── hive-metastore-3.1.3.jar
├── hive-site.xml
├── jackson-datatype-jsr310-2.17.2.jar
├── javax.ws.rs-api-2.0.jar
├── libfb303-0.9.3.jar
├── libthrift-0.9.3.jar
├── log4j-1.2-api-2.17.1.jar
├── log4j-api-2.17.1.jar
├── log4j-core-2.17.1.jar
├── log4j-slf4j-impl-2.17.1.jar
└── mysql-connector-java-5.1.18.jar
EXECUTE CDCSOURCE cdc_kafka_one WITH (
 'connector' = 'mysql-cdc',
 'hostname' = '10.12.51.22',
 'port' = '3306',
 'username' = 'root',
 'password' = 'root',
 'checkpoint' = '3000',
 'scan.startup.mode' = 'initial',
 'parallelism' = '1',
 'table-name' = 'dzh\..*',
 'sink.connector' = 'print'
)

Anything else

No response

Version

1.0.0

Are you willing to submit PR?

Code of Conduct

github-actions[bot] commented 1 week ago

Hello @diaozhonghao, this issue is about CDC/CDCSOURCE, so I assign it to @aiwenmo. If you have any questions, you can comment and reply.

你好 @diaozhonghao, 这个 issue 是关于 CDC/CDCSOURCE 的,所以我把它分配给了 @aiwenmo。如有任何问题,可以评论回复。

Zzm0809 commented 1 week ago

如不需要 doris 相关的整库操作,可暂时将 dinky-cdc-plus 包删除或者移动至其他 dinky 启动时无法加载的路径下

diaozhonghao commented 6 days ago

如不需要 doris 相关的整库操作,可暂时将 dinky-cdc-plus 包删除或者移动至其他 dinky 启动时无法加载的路径下

删除了 但是还是出现同样的错误 image