StarRocks / starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
https://starrocks.io
Apache License 2.0
8.91k stars 1.79k forks source link

[Primary Key]Tablet state will be error if apply phase has not finished and we restart be #9405

Closed caneGuy closed 2 years ago

caneGuy commented 2 years ago

Steps to reproduce the behavior (Required)

  1. load a large dataset
  2. restart be before apply finish

actually this was found when i test with condition update, apply will be slow when i need to read many old rows. in real case we may apply quickly, but i think this is a potential issue

Expected behavior (Required)

tablet state is normal

Real behavior (Required)

ERROR 1064 (HY000): get_applied_rowsets failed, tablet updates is in error state: tablet:23018 actual row size changed after compaction 12121 -> 4095 tablet:23018 #version:5 [6 7.3@4 7.3] pending: rowsets:1
  568 [seg:2 row:12121 del:8026 bytes:1099563 compaction:36095279]

StarRocks version (Required)

caneGuy commented 2 years ago

this was caused by https://github.com/StarRocks/starrocks/pull/8468

caneGuy commented 2 years ago
create database test;
use test;

CREATE TABLE criteo
(
    int12 INT,
    clicked TINYINT,
    int1 INT,
    int2 INT,
    int3 INT,
    int4 INT,
    int5 INT,
    int6 INT,
    int7 INT,
    int8 INT,
    int9 INT,
    int10 INT,
    int11 INT,
    int13 INT,
    icat1 String,
    icat2 String,
    icat3 String,
    icat4 String,
    icat5 String,
    icat6 String,
    icat7 String,
    icat8 String,
    icat9 String,
    icat10 String,
    icat11 String,
    icat12 String,
    icat13 String,
    icat14 String,
    icat15 String,
    icat16 String,
    icat17 String,
    icat18 String,
    icat19 String,
    icat20 String,
    icat21 String,
    icat22 String,
    icat23 String,
    icat24 String,
    icat25 String,
    icat26 String
) ENGINE = OLAP
PRIMARY KEY(int12)
DISTRIBUTED BY HASH(int12) BUCKETS 1
PROPERTIES (
    "replication_num" = "3"
);

curl --location-trusted -u root: -T tmp_1000  -H "label:sr_load_uncompressed_x22" -H "max_filter_ratio:0.2" -H "merge_condition:int1" -H "columns:clicked,int1,int2,int3,int4,int5,int6,int7,int8,int9,int10,int11,int12,int13,icat1,icat2,icat3,icat4,icat5,icat6,icat7,icat8,icat9,icat10,icat11,icat12,icat13,icat14,icat15,icat16,icat17,icat18,icat19,icat20,icat21,icat22,icat23,icat24,icat25,icat26" http://127.0.0.1:8031/api/test/criteo/_stream_load

curl --location-trusted -u root: -T tmp_400000  -H "label:sr_load_uncompressed_x23" -H "max_filter_ratio:0.2" -H "merge_condition:int1" -H "columns:clicked,int1,int2,int3,int4,int5,int6,int7,int8,int9,int10,int11,int12,int13,icat1,icat2,icat3,icat4,icat5,icat6,icat7,icat8,icat9,icat10,icat11,icat12,icat13,icat14,icat15,icat16,icat17,icat18,icat19,icat20,icat21,icat22,icat23,icat24,icat25,icat26" http://127.0.0.1:8031/api/test/criteo/_stream_load

curl --location-trusted -u root: -T tmp_1000000  -H "label:sr_load_uncompressed_x24" -H "max_filter_ratio:0.2" -H "merge_condition:int1" -H "columns:clicked,int1,int2,int3,int4,int5,int6,int7,int8,int9,int10,int11,int12,int13,icat1,icat2,icat3,icat4,icat5,icat6,icat7,icat8,icat9,icat10,icat11,icat12,icat13,icat14,icat15,icat16,icat17,icat18,icat19,icat20,icat21,icat22,icat23,icat24,icat25,icat26" http://127.0.0.1:8031/api/test/criteo/_stream_load

curl --location-trusted -u root: -T tmp  -H "label:sr_load_uncompressed_xn2" -H "max_filter_ratio:0.2" -H "merge_condition:int1" -H "columns:clicked,int1,int2,int3,int4,int5,int6,int7,int8,int9,int10,int11,int12,int13,icat1,icat2,icat3,icat4,icat5,icat6,icat7,icat8,icat9,icat10,icat11,icat12,icat13,icat14,icat15,icat16,icat17,icat18,icat19,icat20,icat21,icat22,icat23,icat24,icat25,icat26" http://127.0.0.1:8031/api/test/criteo/_stream_load