datavane / tis

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI
https://tis.pub
Apache License 2.0
940 stars 211 forks source link

将flink-cdc的MySQL校验MySqlValidator.checkVersion前置 #306

Closed baisui1981 closed 4 months ago

baisui1981 commented 4 months ago

由于mysql本身的原因在flink运行时校验出错

skipUpdateBeforeEvent -> sink: order customer adress) (1/1)#8 (76caadbd101f8365d84ef4a6e8487a) switched frOm INITIALIZING tO FAIL 'failure cause: org.apache.flink,table.api.validationxception: Currently flink Mysgl coc conector only supports Mysgl whose version is
larger or equal to 5.7,but actual is 5.6.at com.ververica.cdc.connectors.mysgl.MySqlValidator.checkVersion(MysglValidator .java:111)at com.ververica.cdc.connectors.mysgl.MysalValidator,validate(MysalValidator,iava:73)at com.ververica.cdc.debezium.DebeziumSourceFunction,open(DebeziumSourceFunction.iava:215)at org.apache.flink.api.common.functions.util.functionUtils.openFunction(FunctionUtils.java:34)at org.apache.flink.streaming.api.operators.AbstractUdfstreamoperator,open(AbstractUdfstreamoperator.java:102)at org.apache.flink.streaming.runtime.tasks.0peratorchain.initializestateAndopenoperators(0peratorchain.java:437)at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:582)at org.apache.flink.streaming.runtime.tasks.streamTaskActionExecutor$synchronizedstreamTaskActionExecutor.call(StreamTaskActionExecutor.java:100)

入口:com.ververica.cdc.connectors.mysgl.MySqlValidator.checkVersion

可以将以上MySqlValidator.checkVersion 这段对MySQL 数据库的校验放置到 com.qlangtech.tis.plugins.incr.flink.cdc.mysql.FlinkCDCMySQLSourceFactory 表单验证逻辑中进行校验

baisui1981 commented 4 months ago

已经添加