pingcap / tispark

TiSpark is built for running Apache Spark on top of TiDB/TiKV
Apache License 2.0
883 stars 244 forks source link

[BUG] 怎么设置一次只提交多条数据? #2691

Closed melin closed 1 year ago

melin commented 1 year ago

怎么设置一次只提交提交记录?看日志一次commit 一条记录,比较慢 image

image

另外遇到一个问题,就是数据写入完成,连接spark 程序不能退出来: image

xuanyu66 commented 1 year ago

Which version of TiSpark?

melin commented 1 year ago

Which version of TiSpark? https://github.com/melin/datatunnel/blob/master/examples/src/main/java/com/superior/datatunnel/examples/DataTunnelMysql2TidbDemo.java

<dependency>
<groupId>com.pingcap.tispark</groupId>
<artifactId>tispark-assembly-3.3_2.12</artifactId>
<version>3.2.0</version>
<exclusions>
<exclusion>
<groupId>org.apache.spark</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.hadoop</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
xuanyu66 commented 1 year ago

A known bug. https://github.com/pingcap/tispark/issues/2633 Maybe try 3.2.1-RC1