search

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.35k stars 928 forks source link

[Bug] MysqlCdcTimestampExtractor.extractTimestamp throws ClassCastException #3535

Closed xingqing closed 2 months ago

xingqing commented 3 months ago

Search before asking

Paimon version

0.8.0 & 0.8.1

Compute Engine

  1. flink 1.18.1
  2. flink-sql-connector-mysql-cdc-2.4.2

Minimal reproduce step

1.Use mysql_sync_database by paimon-flink-action-0.8.0.jar, open tag automatic create, and set table_conf tag.automatic-creation=watermark
2.Submit this job

It always throws : java.lang.ClassCastException: class java.lang.String cannot be cast to class org.apache.paimon.shade.jackson2.com.fasterxml.jackson.databind.JsonNode (java.lang.String is in module java.base of loader 'bootstrap'; org.apache.paimon.shade.jackson2.com.fasterxml.jackson.databind.JsonNode is in unnamed module of loader 'app') at org.apache.paimon.flink.action.cdc.watermark.CdcTimestampExtractorFactory$MysqlCdcTimestampExtractor.extractTimestamp(CdcTimestampExtractorFactory.java:117)__

What doesn't meet your expectations?

this job don't right running and create tag by watermark ， because when I set tag.automatic-creation=process-time， It executes correctly

Anything else?

I think the reason for the exception may be related to these classes CdcDebeziumDeserializationSchema CdcJsonDeserializationSchema

Also, in CdcTimestampExtractorFactory , can we not explicitly rely on unused CDC classes, for example MongoDBSource,PulsarSource,KafkaSource

Are you willing to submit a PR?