[Bug] [Paimon Connector] Paimon Sink Connector Can't Receive Data While Using Sql Transform

[SinyoWong]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

Paimon Sink Connector Can't Receive Data While Using Sql Transform

SeaTunnel Version

2.3.2

SeaTunnel Config

env {
  # You can set flink configuration here
  execution.parallelism = 1
  job.mode = "BATCH"
  #execution.checkpoint.interval = 10000
  #execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint"
}

source {
  # This is a example source plugin **only for test and demonstrate the feature source plugin**
      Paimon {
        warehouse = "hdfs:///data/paimon"
        database = "default"
        table = "test_sink_jdbc"
        hdfs_site_path = "/etc/hadoop/3.1.4.0-315/0/hdfs-site.xml"
        result_table_name = "fake"
      }
  # If you would like to get more information about how to configure seatunnel and see full list of source plugins,
  # please go to https://seatunnel.apache.org/docs/category/source-v2
}

transform {
  Sql {
    source_table_name = "fake"
    result_table_name = "fake1"
    query = "select id,name from fake where id >10"
  }
  # If you would like to get more information about how to configure seatunnel and see full list of transform plugins,
  # please go to https://seatunnel.apache.org/docs/category/transform
}

sink {
  Paimon {
        warehouse = "hdfs:///data/paimon"
        database = "default"
        table = "test_sink_jdbc_2"
        hdfs_site_path = "/etc/hadoop/3.1.4.0-315/0/hdfs-site.xml"
    }
  # If you would like to get more information about how to configure seatunnel and see full list of sink plugins,
  # please go to https://seatunnel.apache.org/docs/category/sink-v2
}

Running Command

./bin/start-seatunnel-flink-13-connector-v2.sh -m yarn-cluster -ynm seatunnel --config ./dev_paimon_to_paimon.conf

Error Exception

No error! Task succeed!

Flink or Spark Version

flink 1.14.5

Java or Scala Version

java 1.8 scala 2.12

Screenshots

Source Paimon table Result : Target Paimon table Result : Target Mysql table Result :

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[SinyoWong]

目前定位到原因是使用sql transform后，paimon没有走到commit方法

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.