apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.81k stars 1.76k forks source link

[Bug] [SQL Server Sink] the writer is very slow #7550

Open RaymondFishWang opened 2 weeks ago

RaymondFishWang commented 2 weeks ago

Search before asking

What happened

I try sqlserver sink, the speed is very slow. The data volumn is 36w, if I use different source ,the result is the same. If i use mysql sink, we can load the whole data in 20s. And I watch the result table for sqlserver , we only insert 1k data into the table in 1s. It seems the batch size doesn't work. So the sqlserver sink has some problem .I also tried different jar lib.

SeaTunnel Version

2.3.5

SeaTunnel Config

sink {
    Jdbc {
        url = "jdbc:sqlserver://***:1433;DatabaseName=***;encrypt=false"
        driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
        database="***"
        user = "****"
        password = "***"
        data_save_mode="DROP_DATA"
        schema_save_mode="ERROR_WHEN_SCHEMA_NOT_EXIST"
        generate_sink_sql = true
        batch_size = 200000
        table = "dbo.****"
        }
}

Running Command

/data/share_storage/seatunnel-2.3.5/bin/seatunnel.sh --config  **.config

Error Exception

no exception

Zeta or Flink or Spark Version

zeta

Java or Scala Version

1.8

Screenshots

image

Are you willing to submit PR?

Code of Conduct