StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.63k stars 1.74k forks source link

[shared-data] storage cleanup due to incomplete data ingestion transaction. #48334

Open zhbdesign opened 1 month ago

zhbdesign commented 1 month ago

Enhancement

通过insert into select * from demo进行插入数据,中间因为超时停止,minio数据已经存在1.8G,但是表里为空,这符合Starrocks的事务原理,但是minio上的1.8G数据什么时候会自动清理什么参数进行控制的,可以手动执行什么sql进行清理吗,等了半小时还是没有自动删除

Inserting data through insert into select * from demo stopped due to timeout, and minio data already exists 1.8G, but the table is empty, which is in line with Starrocks' transaction principle. However, when does the 1.8G data on minio automatically clean up, what parameters are controlled, and can manual SQL be executed for cleaning? After waiting for half an hour, it still hasn't been automatically deleted

zhbdesign commented 1 month ago

set query_timeout =10.; insert into demo_table select * from demo_table1;即可复现

zhbdesign commented 1 month ago

It hasn't been deleted yet, so it shouldn't be automatically deleted. It's been a long time

srlch commented 1 month ago

@zhbdesign What is the version of StarRocks tested?

zhbdesign commented 1 month ago

@zhbdesign What is the version of StarRocks tested?

3.3.0