[X] I had searched in the issues and found no similar issues.
What happened
rest api:/hazelcast/rest/maps/finished-jobs,The SinkWriteCount returned is inaccurate, Always the same as SourceReceivedCount.
During testing, some duplicate data was created, The source table has 13 data entries, of which 1 is duplicated.
The target table ID serves as the primary key. After executing the seatunnel task, 12 data entries were written to the target table, and the other 1 primary key was duplicated, resulting in a write failure.
But the SinkWriteCount returned by the API interface above is 13. How should I set it to ensure that the SinkWriteCount is correct.
SeaTunnel Version
2.3.7
SeaTunnel Config
env {
job.mode = "BATCH"
job.name = "data_label"
job.retry.times=0
parallelism = 1
}
source {
Jdbc {
url= "jdbc:mysql://xxx:3306/test?serverTimezone=GMT%2B8&characterEncoding=UTF-8&useUnicode=true&useSSL=false"
driver= "com.mysql.jdbc.Driver"
user = "root"
password = "xxx"
result_table_name= "data_label_src"
query= "SELECT id as id, label_name AS label_name,order_no as order_no,is_del AS is_del,crt_time AS crt_time,crt_user AS crt_user,crt_name AS crt_name,crt_host AS crt_host,upd_time AS upd_time,upd_user AS upd_user,upd_name AS upd_name,upd_host AS upd_host,description AS description FROM data_label"
}
}
sink {
Jdbc {
url= "jdbc:mysql://xxxx:3306/test1?serverTimezone=GMT%2B8&characterEncoding=UTF-8&useUnicode=true&useSSL=false"
driver= "com.mysql.jdbc.Driver"
user = "root"
password = "xxxx"
source_table_name = "data_label_src"
database = "test1"
schema_save_mode = "ERROR_WHEN_SCHEMA_NOT_EXIST"
max_retries=0
table = "data_label"
generate_sink_sql = true
}
}
Search before asking
What happened
rest api:/hazelcast/rest/maps/finished-jobs,The SinkWriteCount returned is inaccurate, Always the same as SourceReceivedCount. During testing, some duplicate data was created, The source table has 13 data entries, of which 1 is duplicated. The target table ID serves as the primary key. After executing the seatunnel task, 12 data entries were written to the target table, and the other 1 primary key was duplicated, resulting in a write failure. But the SinkWriteCount returned by the API interface above is 13. How should I set it to ensure that the SinkWriteCount is correct.
SeaTunnel Version
2.3.7
SeaTunnel Config
Running Command
Error Exception
Zeta or Flink or Spark Version
No response
Java or Scala Version
java-1.8.0_412
Screenshots
No response
Are you willing to submit PR?
Code of Conduct