apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
8.07k stars 1.83k forks source link

[Bug] [Module Name] pg写到doris时读取超出源数据量 #7932

Open gaoxiaolv opened 4 weeks ago

gaoxiaolv commented 4 weeks ago

Search before asking

What happened

image image

SeaTunnel Version

2.3.8

SeaTunnel Config

{
    "transform" : [],
    "sink" : [
        {
            "password" : "",
            "database" : "",
            "fenodes" : "",
            "sink.enable-2pc" : "true",
            "doris.config" : {
                "read_json_by_line" : "true",
                "format" : "json"
            },
            "plugin_name" : "Doris",
            "sink.label-prefix" : "test_json",
            "table" : "table_incident_sink",
            "username" : ""
        }
    ],
    "source" : [
        {
            "password" : "",
            "driver" : "org.postgresql.Driver",
            "query" : "select * from dw_incident order by incidentid limit 2800000",
            "plugin_name" : "Jdbc",
            "user" : "",
            "url" : ""
        }
    ],
    "env" : {
        "job.mode" : "BATCH",
        "parallelism" : 8
    }
}

Running Command

./seatunnel.sh --config /home/iuopadmin/test.conf

Error Exception

pg写到doris时读取超出源数据量

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct