StarRocks / starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
https://starrocks.io
Apache License 2.0
8.9k stars 1.78k forks source link

When performing concurrent insert operations, a "user cancel" error occurred. #50050

Open zhangle1250 opened 2 months ago

zhangle1250 commented 2 months ago

Steps to reproduce the behavior (Required)

  1. CREATE TABLE '...' CREATE TABLE MV_LEAF_EARLIER_SINGLE_AGG_3aa5cd80 ( 4137a775e0654dd98125f6c8939e52ad varchar(255) NULL COMMENT "", dt date NULL COMMENT "", 120ef87795504884a0b3f453a062276a double NULL COMMENT "" ) ENGINE=OLAP DUPLICATE KEY(4137a775e0654dd98125f6c8939e52ad) PARTITION BY RANGE(dt) (PARTITION p20230101 VALUES [("2023-01-01"), ("2023-01-02")), PARTITION p20230102 VALUES [("2023-01-02"), ("2023-01-03")), PARTITION p20230103 VALUES [("2023-01-03"), ("2023-01-04")), PARTITION p20230104 VALUES [("2023-01-04"), ("2023-01-05")), PARTITION p20230105 VALUES [("2023-01-05"), ("2023-01-06")), PARTITION p20230106 VALUES [("2023-01-06"), ("2023-01-07")), PARTITION p20230107 VALUES [("2023-01-07"), ("2023-01-08")), PARTITION p20230108 VALUES [("2023-01-08"), ("2023-01-09"));

  2. The following code is executed concurrently:

public class Test1 { public static void main(String[] args) throws SQLException, ClassNotFoundException, InterruptedException {

    Class.forName("com.mysql.jdbc.Driver");

    ThreadPoolExecutor THREAD_POOL_EXECUTOR = new ThreadPoolExecutor(4, 4,
            3, TimeUnit.SECONDS, new LinkedBlockingDeque<>(1024));
    for (int i = 0; i < 20; i++) {
        LocalDate of = LocalDate.of(2024, 1, 1);
        of = of.plusDays(i-1);
        String date1 = of.format(DateTimeFormatter.ofPattern("yyyy-MM-dd"));
        String p1 = of.format(DateTimeFormatter.ofPattern("yyyyMMdd"));
        LocalDate localDate = of.plusDays(i+1);
        String date2 = localDate.format(DateTimeFormatter.ofPattern("yyyy-MM-dd"));
        String p2 = localDate.format(DateTimeFormatter.ofPattern("yyyyMMdd"));

        String selectSql = "INSERT  OVERWRITE `mydb`.`MV_LEAF_EARLIER_SINGLE_AGG_3aa5cd80` PARTITION('p20240101')   SELECT  "
                           + "'abc','20240101',1.86 from mydb.dual\n";
        System.err.println(new Date());
        Future<?> submit = THREAD_POOL_EXECUTOR.submit(() -> {
            Statement statement = null;
            Connection connection = null;
            try {
                connection = DriverManager.getConnection("jdbc:mysql:// 10.5.20.100:9030 ", "root", "xxx");
                statement = connection.createStatement();
                statement.execute("SET enable_insert_strict = false;");
                statement = connection.createStatement();
                statement.execute(selectSql);
                System.out.println(1);
            } catch (SQLException sqlException) {
                sqlException.printStackTrace();
            }finally {
                try {
                    connection.close();
                } catch (SQLException sqlException) {
                    sqlException.printStackTrace();
                }
            }
        });

    }

    Thread.sleep(3);
    System.out.println(2);
}

Expected behavior (Required)

The program executes correctly.

Real behavior (Required)

image

2024-08-20 21:46:58.332+08:00 WARN (analyze-task-concurrency-pool-0|485) [StatisticExecutor.collectStatistics():285] Collect statistics error com.starrocks.sql.analyzer.SemanticException: Getting analyzing error. Detail message: Unknown partition 'p20240101_7087647' in table 'MV_LEAF_EARLIER_SINGLE_AGG_3aa5cd80'. at com.starrocks.sql.analyzer.QueryAnalyzer.resolveTable(QueryAnalyzer.java:1056) ~[starrocks-fe.jar:?] at com.starrocks.sql.analyzer.QueryAnalyzer.access$200(QueryAnalyzer.java:99) ~[starrocks-fe.jar:?] at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.resolveTableRef(QueryAnalyzer.java:298) ~[starrocks-fe.jar:?] at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:196) ~[starrocks-fe.jar:?] at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:116) ~[starrocks-fe.jar:?] at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:242) ~[starrocks-fe.jar:?] at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:121) ~[starrocks-fe.jar:?] at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryRelation(

StarRocks version (Required)

Seaven commented 1 month ago

I think reason is from mydb.dual, sr don't support db.dual, you can try from dual not from mydb.dual.

BTW, the stack will happened when concurrent insert overwrite partition, but it's don't effect use, it's an internal task that can fail