matrixorigin / matrixone

Hyperconverged cloud-edge native database
https://docs.matrixorigin.cn/en
Apache License 2.0
1.77k stars 275 forks source link

[Bug]: Drop table that is not referenced by snapshots will not be GC #16997

Closed LeftHandCold closed 3 months ago

LeftHandCold commented 3 months ago

Is there an existing issue for the same bug?

Branch Name

1.2-dev

Commit ID

ffd38a22e3c04dda4dc14a917ee6837fadb0e6ca

Other Environment Information

- Hardware parameters:
- OS type:
- Others:

Actual Behavior

Drop table that is not referenced by snapshots。data is will not be GC

Expected Behavior

No response

Steps to Reproduce

create table1 and load data
create snapshot
create table2 and load data
drop table2

Additional information

No response

LeftHandCold commented 3 months ago

fixed

Ariznawlll commented 3 months ago

1.2-dev commit: 4df8e561b3a6f36ab519d19f69d55956bab90955

测试前准备: 单机环境在tn.toml里面加上配置:

[tn.Ckp]
flush-interval = "10s"
min-count = 100
scan-interval = "5s"
incremental-interval = "30s"
global-min-count = 9

[tn.GCCfg]
gc-ttl = "5m"
scan-gc-interval = "2m"
check-gc = true

并对应修改代码配置:(收集5min中内的过期数据) /Users/ariznawl/matrixone/pkg/vm/engine/tae/options/types.go

image
具体步骤:
create table_with_pk_index_for_load_100m and load data;
create snapshot sp01 for account sys;
create table t1(a int primary key);
insert into t1 (select * from generate_series(1,1000000,1)g);
drop table t1;

create snapshot sp01 for account sys;

image

查看存储t1表所在的block信息:

image

删除表t1后查看表t1所在的存储block:

image

main commit:eb1f0f578faab5fc7f977c23a3a99b9e62580186

测试步骤:

create table_with_pk_index_for_load_100m and load data;
create snapshot sp01 for account sys;
create table t1(a int primary key);
insert into t1 (select * from generate_series(1,1000000,1)g);
insert into t1 (select * from generate_series(1000001,10000000,1)g);
drop table t1;

查看存储t1表所在的block信息:

image

删除表后,查看t1表所在的block信息:

image

测试通过。