Open habma1 opened 1 year ago
Bug Description Java 使用参数绑定方式对单表每次写入一条数据,预计要写入100w条数据,在写入几十万条数据时,Java 程序进程无异常抛出便被强行终止
Screenshots
Environment
Additional Context 刚开始模拟对一百万张表每张表插入1000-50000条数据出现该问题,后发现在写入操作达到一定次数之后(几十万次)Java程序会被终止。复现问题简化版本程序 `package com.example;
import com.taosdata.jdbc.TSDBPreparedStatement;
import java.sql.Connection; import java.sql.DriverManager; import java.sql.SQLException; import java.sql.Statement; import java.util.ArrayList;
public class StmtInsert { private static final int RECORDS_PER_TABLE = 600_000;
public static void main(String[] args) throws SQLException { insertData(); } private static void insertData() throws SQLException { try (final Connection conn = getConnection()) { createTable(conn); final String psql = "INSERT INTO ? USING meters TAGS(?, ?) VALUES(?, ?, ?, ?)"; try (final TSDBPreparedStatement pst = (TSDBPreparedStatement) conn.prepareStatement(psql)) { System.out.println("-> 开始插入数据 <-"); int counter = 0; for (int i = 0; i < RECORDS_PER_TABLE; i++) { pst.clearBatch(); final String line = getRowData(i); final String[] ps = line.split(","); // bind table name and tags pst.setTableName(ps[0]); pst.setTagString(0, ps[5]); pst.setTagInt(1, Integer.valueOf(ps[6])); // bind values pst.setTimestamp(0, toArray(Long.valueOf(ps[1]))); pst.setFloat(1, toArray(Float.valueOf(ps[2]))); pst.setInt(2, toArray(Integer.valueOf(ps[3]))); pst.setFloat(3, toArray(Float.valueOf(ps[4]))); pst.columnDataAddBatch(); pst.columnDataExecuteBatch(); counter++; if (counter % 5000 == 0) { System.out.println("------------已插入数据 " + counter); } } System.out.println("执行完毕"); } } } private static Connection getConnection() throws SQLException { System.out.println("-> 建立连接 <-"); String jdbcUrl = "jdbc:TAOS://localhost:6030?user=root&password=taosdata"; return DriverManager.getConnection(jdbcUrl); } private static void createTable(Connection conn) throws SQLException { System.out.println("-> 创建数据库和超级表 <-"); try (Statement stmt = conn.createStatement()) { stmt.execute("DROP DATABASE IF EXISTS power"); stmt.execute("CREATE DATABASE IF NOT EXISTS power KEEP 3650"); stmt.executeUpdate("USE power"); stmt.execute("CREATE STABLE IF NOT EXISTS meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) " + "TAGS (location BINARY(64), groupId INT)"); } } private static String getRowData(int i) { long time = 1691478964333L + i; return "d0," + time + ",10.30000,219,0.31000,California.SanFrancisco,2"; } private static <T> ArrayList<T> toArray(T v) { ArrayList<T> result = new ArrayList<>(); result.add(v); return result; }
}`
监控一下内存占用是不是资源没释放被系统杀掉了?
NSERT INTO ? USING meters TAGS
SQL拼接的方式插入,会不会有SQL注入的风险呢?taos-jdbcdriver是不是内部做了优化呢?
可能是taos.dll的资源泄漏没有释放引起的,已提过issue,但还未修复 https://github.com/taosdata/TDengine/issues/22599
Bug Description Java 使用参数绑定方式对单表每次写入一条数据,预计要写入100w条数据,在写入几十万条数据时,Java 程序进程无异常抛出便被强行终止
Screenshots
Environment
Additional Context 刚开始模拟对一百万张表每张表插入1000-50000条数据出现该问题,后发现在写入操作达到一定次数之后(几十万次)Java程序会被终止。复现问题简化版本程序 `package com.example;
import com.taosdata.jdbc.TSDBPreparedStatement;
import java.sql.Connection; import java.sql.DriverManager; import java.sql.SQLException; import java.sql.Statement; import java.util.ArrayList;
public class StmtInsert { private static final int RECORDS_PER_TABLE = 600_000;
}`