search

apache / flink-cdc

Flink CDC is a streaming data integration tool
https://nightlies.apache.org/flink/flink-cdc-docs-stable
Apache License 2.0
5.46k stars 1.83k forks source link

[Bug] use flink-connector-mysql-cdc 2.1.1 ， the primary key of mysql is Binary。 Get an error: [B cannot be cast to java.lang.Comparab #2746

Closed chenxu2656 closed 7 months ago

chenxu2656 commented 7 months ago

Search before asking

Flink version

1.13.5

Flink CDC version

2.1.1

Database and its version

Mysql 8

Minimal reproduce step

    MySqlSource<String> mySqlSource = MySqlSource.<String>builder()
            .hostname("127.0.0.1")
            .port(3306)
            .databaseList("test") // monitor all tables under inventory database
            .tableList("test.test") // set captured table
            .username("root")
            .password("123456")
            .deserializer(new JsonDebeziumDeserializationSchema()) // converts SourceRecord to JSON String
            .build();

mysql primary key is Binary

What did you expect to see?

not get error

What did you see instead?

[SourceCoordinator-Source: MySQL Source -> Sink: Print to Std. Out] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL Source -> Sink: Print to Std. Out while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover.
org.apache.flink.util.FlinkRuntimeException: Generate Splits for table test.test error
    at com.ververica.cdc.connectors.mysql.source.assigners.ChunkSplitter.generateSplits(ChunkSplitter.java:115)
    at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:168)
    at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:113)
    at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:167)
    at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:98)
    at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172)
    at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassCastException: [B cannot be cast to java.lang.Comparable
    at com.ververica.cdc.connectors.mysql.source.utils.ObjectUtils.compare(ObjectUtils.java:86)
    at com.ververica.cdc.connectors.mysql.source.assigners.ChunkSplitter.nextChunkEnd(ChunkSplitter.java:239)
    at com.ververica.cdc.connectors.mysql.source.assigners.ChunkSplitter.splitUnevenlySizedChunks(ChunkSplitter.java:208)
    at com.ververica.cdc.connectors.mysql.source.assigners.ChunkSplitter.splitTableIntoChunks(ChunkSplitter.java:162)
    at com.ververica.cdc.connectors.mysql.source.assigners.ChunkSplitter.generateSplits(ChunkSplitter.java:85)
    ... 9 more

Anything else?

No response

Are you willing to submit a PR?

leonardBang commented 7 months ago

try to bump cdc version to 2.4.2, IIRC, this issue has been fixed

chenxu2656 commented 7 months ago 
package mysql.connect;
import com.ververica.cdc.connectors.mysql.source.MySqlSource;
import com.ververica.cdc.debezium.JsonDebeziumDeserializationSchema;
import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import mysql.connect.serialization.CustomJsonDebeziumDeserializationSchema;
import java.util.Properties;

public class MySqlCdcPrint {

    public static void main(String[] args) throws Exception {
        MySqlSource<String> mySqlSource = MySqlSource.<String>builder()
                .hostname("127.0.0.1")
                .port(3306)
                .databaseList("test") // monitor all tables under inventory database
                .tableList("test.tge") // set captured table
                .username("root")
                .password("123456")
                .deserializer(new JsonDebeziumDeserializationSchema()) // converts SourceRecord to JSON String
                .build();

        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        // enable checkpoint
        env.enableCheckpointing(60000);  //checkpoint需要什么条件?com/ververica/cdc/connectors/mysql/source/enumerator/MySqlSourceEnumerator.snapshotState()
        env
                .fromSource(mySqlSource, WatermarkStrategy.noWatermarks(), "MySQL Source")
                // set 4 parallel source tasks
                .setParallelism(4)
                .print("插入数据===>").setParallelism(1); // use parallelism 1 for sink to keep message ordering
        env.execute("MySqlCdcPrint1");
    }
}

*thx, but after bump to 2.4.2 ， flink version is 1.13.5 . I checked the cdc document, seems cdc2.4 support flink 1.13

get this error**

/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/bin/java -javaagent:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=57027:/Applications/IntelliJ IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -classpath /Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/charsets.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/cldrdata.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/dnsns.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/jaccess.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/jfxrt.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/localedata.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/nashorn.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/sunec.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/ext/zipfs.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/jce.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/jfr.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/jfxswt.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/jsse.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/management-agent.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/resources.jar:/Users/xuchen/Library/Java/JavaVirtualMachines/corretto-1.8.0_392/Contents/Home/jre/lib/rt.jar:/Users/xuchen/Desktop/aitmed/esflink/cdcJob/target/classes:/Users/xuchen/.m2/repository/com/ververica/flink-connector-mysql-cdc/2.4.2/flink-connector-mysql-cdc-2.4.2.jar:/Users/xuchen/.m2/repository/com/ververica/flink-connector-debezium/2.4.2/flink-connector-debezium-2.4.2.jar:/Users/xuchen/.m2/repository/io/debezium/debezium-api/1.9.7.Final/debezium-api-1.9.7.Final.jar:/Users/xuchen/.m2/repository/io/debezium/debezium-embedded/1.9.7.Final/debezium-embedded-1.9.7.Final.jar:/Users/xuchen/.m2/repository/org/apache/kafka/connect-api/3.2.0/connect-api-3.2.0.jar:/Users/xuchen/.m2/repository/org/apache/kafka/kafka-clients/3.2.0/kafka-clients-3.2.0.jar:/Users/xuchen/.m2/repository/javax/ws/rs/javax.ws.rs-api/2.1.1/javax.ws.rs-api-2.1.1.jar:/Users/xuchen/.m2/repository/org/apache/kafka/connect-runtime/3.2.0/connect-runtime-3.2.0.jar:/Users/xuchen/.m2/repository/org/apache/kafka/connect-transforms/3.2.0/connect-transforms-3.2.0.jar:/Users/xuchen/.m2/repository/org/apache/kafka/kafka-tools/3.2.0/kafka-tools-3.2.0.jar:/Users/xuchen/.m2/repository/net/sourceforge/argparse4j/argparse4j/0.7.0/argparse4j-0.7.0.jar:/Users/xuchen/.m2/repository/ch/qos/reload4j/reload4j/1.2.19/reload4j-1.2.19.jar:/Users/xuchen/.m2/repository/org/bitbucket/b_c/jose4j/0.7.9/jose4j-0.7.9.jar:/Users/xuchen/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.12.6/jackson-annotations-2.12.6.jar:/Users/xuchen/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-json-provider/2.12.6/jackson-jaxrs-json-provider-2.12.6.jar:/Users/xuchen/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-base/2.12.6/jackson-jaxrs-base-2.12.6.jar:/Users/xuchen/.m2/repository/com/fasterxml/jackson/module/jackson-module-jaxb-annotations/2.12.6/jackson-module-jaxb-annotations-2.12.6.jar:/Users/xuchen/.m2/repository/jakarta/xml/bind/jakarta.xml.bind-api/2.3.2/jakarta.xml.bind-api-2.3.2.jar:/Users/xuchen/.m2/repository/jakarta/activation/jakarta.activation-api/1.2.1/jakarta.activation-api-1.2.1.jar:/Users/xuchen/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet/2.34/jersey-container-servlet-2.34.jar:/Users/xuchen/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet-core/2.34/jersey-container-servlet-core-2.34.jar:/Users/xuchen/.m2/repository/org/glassfish/hk2/external/jakarta.inject/2.6.1/jakarta.inject-2.6.1.jar:/Users/xuchen/.m2/repository/jakarta/ws/rs/jakarta.ws.rs-api/2.1.6/jakarta.ws.rs-api-2.1.6.jar:/Users/xuchen/.m2/repository/org/glassfish/jersey/inject/jersey-hk2/2.34/jersey-hk2-2.34.jar:/Users/xuchen/.m2/repository/org/glassfish/hk2/hk2-locator/2.6.1/hk2-locator-2.6.1.jar:/Users/xuchen/.m2/repository/org/glassfish/hk2/external/aopalliance-repackaged/2.6.1/aopalliance-repackaged-2.6.1.jar:/Users/xuchen/.m2/repository/org/glassfish/hk2/hk2-api/2.6.1/hk2-api-2.6.1.jar:/Users/xuchen/.m2/repository/org/glassfish/hk2/hk2-utils/2.6.1/hk2-utils-2.6.1.jar:/Users/xuchen/.m2/repository/javax/xml/bind/jaxb-api/2.3.0/jaxb-api-2.3.0.jar:/Users/xuchen/.m2/repository/javax/activation/activation/1.1.1/activation-1.1.1.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-server/9.4.44.v20210927/jetty-server-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-http/9.4.44.v20210927/jetty-http-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-io/9.4.44.v20210927/jetty-io-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-servlet/9.4.44.v20210927/jetty-servlet-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-security/9.4.44.v20210927/jetty-security-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-util-ajax/9.4.44.v20210927/jetty-util-ajax-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-servlets/9.4.44.v20210927/jetty-servlets-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-continuation/9.4.44.v20210927/jetty-continuation-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-util/9.4.44.v20210927/jetty-util-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/org/eclipse/jetty/jetty-client/9.4.44.v20210927/jetty-client-9.4.44.v20210927.jar:/Users/xuchen/.m2/repository/org/reflections/reflections/0.9.12/reflections-0.9.12.jar:/Users/xuchen/.m2/repository/org/apache/maven/maven-artifact/3.8.4/maven-artifact-3.8.4.jar:/Users/xuchen/.m2/repository/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.jar:/Users/xuchen/.m2/repository/org/apache/kafka/connect-json/3.2.0/connect-json-3.2.0.jar:/Users/xuchen/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.12.6/jackson-datatype-jdk8-2.12.6.jar:/Users/xuchen/.m2/repository/org/apache/kafka/connect-file/3.2.0/connect-file-3.2.0.jar:/Users/xuchen/.m2/repository/io/debezium/debezium-connector-mysql/1.9.7.Final/debezium-connector-mysql-1.9.7.Final.jar:/Users/xuchen/.m2/repository/io/debezium/debezium-core/1.9.7.Final/debezium-core-1.9.7.Final.jar:/Users/xuchen/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.13.2/jackson-core-2.13.2.jar:/Users/xuchen/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.13.2.2/jackson-databind-2.13.2.2.jar:/Users/xuchen/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.13.2/jackson-datatype-jsr310-2.13.2.jar:/Users/xuchen/.m2/repository/com/google/guava/guava/30.1.1-jre/guava-30.1.1-jre.jar:/Users/xuchen/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar:/Users/xuchen/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/Users/xuchen/.m2/repository/io/debezium/debezium-ddl-parser/1.9.7.Final/debezium-ddl-parser-1.9.7.Final.jar:/Users/xuchen/.m2/repository/org/antlr/antlr4-runtime/4.8/antlr4-runtime-4.8.jar:/Users/xuchen/.m2/repository/com/zendesk/mysql-binlog-connector-java/0.27.2/mysql-binlog-connector-java-0.27.2.jar:/Users/xuchen/.m2/repository/com/github/luben/zstd-jni/1.5.0-2/zstd-jni-1.5.0-2.jar:/Users/xuchen/.m2/repository/mysql/mysql-connector-java/8.0.28/mysql-connector-java-8.0.28.jar:/Users/xuchen/.m2/repository/com/esri/geometry/esri-geometry-api/2.2.0/esri-geometry-api-2.2.0.jar:/Users/xuchen/.m2/repository/com/zaxxer/HikariCP/4.0.3/HikariCP-4.0.3.jar:/Users/xuchen/.m2/repository/org/apache/commons/commons-lang3/3.7/commons-lang3-3.7.jar:/Users/xuchen/.m2/repository/org/slf4j/slf4j-api/1.7.15/slf4j-api-1.7.15.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-shaded-guava/30.1.1-jre-16.1/flink-shaded-guava-30.1.1-jre-16.1.jar:/Users/xuchen/.m2/repository/org/awaitility/awaitility/4.0.1/awaitility-4.0.1.jar:/Users/xuchen/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/Users/xuchen/.m2/repository/com/alibaba/fastjson/1.2.75/fastjson-1.2.75.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-streaming-java_2.12/1.13.5/flink-streaming-java_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-core/1.13.5/flink-core-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-annotations/1.13.5/flink-annotations-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-metrics-core/1.13.5/flink-metrics-core-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-shaded-asm-7/7.1-13.0/flink-shaded-asm-7-7.1-13.0.jar:/Users/xuchen/.m2/repository/com/esotericsoftware/kryo/kryo/2.24.0/kryo-2.24.0.jar:/Users/xuchen/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:/Users/xuchen/.m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/Users/xuchen/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/xuchen/.m2/repository/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-file-sink-common/1.13.5/flink-file-sink-common-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-runtime_2.12/1.13.5/flink-runtime_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-queryable-state-client-java/1.13.5/flink-queryable-state-client-java-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-hadoop-fs/1.13.5/flink-hadoop-fs-1.13.5.jar:/Users/xuchen/.m2/repository/commons-io/commons-io/2.8.0/commons-io-2.8.0.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-shaded-netty/4.1.49.Final-13.0/flink-shaded-netty-4.1.49.Final-13.0.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-shaded-jackson/2.12.1-13.0/flink-shaded-jackson-2.12.1-13.0.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-shaded-zookeeper-3/3.4.14-13.0/flink-shaded-zookeeper-3-3.4.14-13.0.jar:/Users/xuchen/.m2/repository/org/javassist/javassist/3.24.0-GA/javassist-3.24.0-GA.jar:/Users/xuchen/.m2/repository/org/scala-lang/scala-library/2.12.7/scala-library-2.12.7.jar:/Users/xuchen/.m2/repository/com/typesafe/akka/akka-actor_2.12/2.5.21/akka-actor_2.12-2.5.21.jar:/Users/xuchen/.m2/repository/com/typesafe/config/1.3.3/config-1.3.3.jar:/Users/xuchen/.m2/repository/org/scala-lang/modules/scala-java8-compat_2.12/0.8.0/scala-java8-compat_2.12-0.8.0.jar:/Users/xuchen/.m2/repository/com/typesafe/akka/akka-stream_2.12/2.5.21/akka-stream_2.12-2.5.21.jar:/Users/xuchen/.m2/repository/org/reactivestreams/reactive-streams/1.0.2/reactive-streams-1.0.2.jar:/Users/xuchen/.m2/repository/com/typesafe/ssl-config-core_2.12/0.3.7/ssl-config-core_2.12-0.3.7.jar:/Users/xuchen/.m2/repository/org/scala-lang/modules/scala-parser-combinators_2.12/1.1.1/scala-parser-combinators_2.12-1.1.1.jar:/Users/xuchen/.m2/repository/com/typesafe/akka/akka-protobuf_2.12/2.5.21/akka-protobuf_2.12-2.5.21.jar:/Users/xuchen/.m2/repository/com/typesafe/akka/akka-slf4j_2.12/2.5.21/akka-slf4j_2.12-2.5.21.jar:/Users/xuchen/.m2/repository/org/clapper/grizzled-slf4j_2.12/1.3.2/grizzled-slf4j_2.12-1.3.2.jar:/Users/xuchen/.m2/repository/com/github/scopt/scopt_2.12/3.5.0/scopt_2.12-3.5.0.jar:/Users/xuchen/.m2/repository/org/xerial/snappy/snappy-java/1.1.8.3/snappy-java-1.1.8.3.jar:/Users/xuchen/.m2/repository/com/twitter/chill_2.12/0.7.6/chill_2.12-0.7.6.jar:/Users/xuchen/.m2/repository/com/twitter/chill-java/0.7.6/chill-java-0.7.6.jar:/Users/xuchen/.m2/repository/org/lz4/lz4-java/1.6.0/lz4-java-1.6.0.jar:/Users/xuchen/.m2/repository/org/apache/commons/commons-math3/3.5/commons-math3-3.5.jar:/Users/xuchen/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/xuchen/.m2/repository/org/apache/flink/force-shading/1.13.5/force-shading-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-clients_2.12/1.13.5/flink-clients_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-optimizer_2.12/1.13.5/flink-optimizer_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/commons-cli/commons-cli/1.3.1/commons-cli-1.3.1.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-java/1.13.5/flink-java-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-table-planner-blink_2.12/1.13.5/flink-table-planner-blink_2.12-1.13.5-tests.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-table-common/1.13.5/flink-table-common-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-connector-files/1.13.5/flink-connector-files-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-connector-base/1.13.5/flink-connector-base-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-table-api-java/1.13.5/flink-table-api-java-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-table-api-scala_2.12/1.13.5/flink-table-api-scala_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/org/scala-lang/scala-reflect/2.12.7/scala-reflect-2.12.7.jar:/Users/xuchen/.m2/repository/org/scala-lang/scala-compiler/2.12.7/scala-compiler-2.12.7.jar:/Users/xuchen/.m2/repository/org/scala-lang/modules/scala-xml_2.12/1.0.6/scala-xml_2.12-1.0.6.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-table-api-java-bridge_2.12/1.13.5/flink-table-api-java-bridge_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-table-api-scala-bridge_2.12/1.13.5/flink-table-api-scala-bridge_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-scala_2.12/1.13.5/flink-scala_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-streaming-scala_2.12/1.13.5/flink-streaming-scala_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/org/apache/flink/flink-table-runtime-blink_2.12/1.13.5/flink-table-runtime-blink_2.12-1.13.5.jar:/Users/xuchen/.m2/repository/org/codehaus/janino/janino/3.0.11/janino-3.0.11.jar:/Users/xuchen/.m2/repository/org/codehaus/janino/commons-compiler/3.0.11/commons-compiler-3.0.11.jar:/Users/xuchen/.m2/repository/org/apache/calcite/avatica/avatica-core/1.17.0/avatica-core-1.17.0.jar:/Users/xuchen/.m2/repository/org/slf4j/slf4j-simple/1.7.25/slf4j-simple-1.7.25.jar mysql.connect.MySqlCdcPrint
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava18/com/google/common/collect/Lists
    at org.apache.flink.streaming.api.transformations.LegacySinkTransformation.getTransitivePredecessors(LegacySinkTransformation.java:117)
    at org.apache.flink.streaming.api.graph.StreamGraphGenerator.lambda$existsUnboundedSource$0(StreamGraphGenerator.java:410)
    at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
    at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1361)
    at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
    at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
    at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
    at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
    at org.apache.flink.streaming.api.graph.StreamGraphGenerator.existsUnboundedSource(StreamGraphGenerator.java:407)
    at org.apache.flink.streaming.api.graph.StreamGraphGenerator.shouldExecuteInBatchMode(StreamGraphGenerator.java:388)
    at org.apache.flink.streaming.api.graph.StreamGraphGenerator.generate(StreamGraphGenerator.java:297)
    at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:2010)
    at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:1995)
    at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1834)
    at mysql.connect.MySqlCdcPrint.main(MySqlCdcPrint.java:36)
Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.guava18.com.google.common.collect.Lists
    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 19 more
chenxu2656 commented 7 months ago

add： bump flink version to 1.14.0 fix upper error

if get time zoon issues

import java.util.Properties;
 Properties debeziumProperties = new Properties();
 debeziumProperties.setProperty("database.serverTimezone", "UTC");
MySqlSource<String> mySqlSource = MySqlSource.<String>builder()
                ..........
                .debeziumProperties(debeziumProperties)
                .build();