Tencent / TBase

TBase is an enterprise-level distributed HTAP database. Through a single database cluster to provide users with highly consistent distributed database services and high-performance data warehouse services, a set of integrated enterprise-level solutions is formed.
Other
1.38k stars 262 forks source link

写入测试数据时,表的数据分布突然从2个节点变为1个节点,数据无丢失。多次测试都出现。TBase版本2.1 #81

Open softlanguage opened 3 years ago

softlanguage commented 3 years ago

写入测试数据时,表的数据分布突然从2个节点变为1个节点,数据无丢失。多次测试都出现。TBase版本2.1

select node_name,node_type,node_host,node_port, * from pgxc_node where node_type='D';

image

SELECT xc_node_id, count(*) FROM t1_pt GROUP BY xc_node_id;

image

-- 建表
create table public.t1_pt
(
f1 int not null,
f2 timestamp not null,
f3 varchar(50),
primary key(f1)
) 
distribute by shard(f1) 
to group default_group;

-- 写入数据
insert into t1_pt (f1, f2, f3)
SELECT generate_series(1001,200000) as f1,
now() as f2, repeat('a',(random()*25)::integer) as order_info;
q2683252 commented 3 years ago

表的数据分布突然从2个节点变为1个节点

这个是怎么确定出来的?

softlanguage commented 3 years ago

环境:2台服务器;

————1 create table public.t1_pt ( f1 int not null, f2 timestamp not null, f3 varchar(50), primary key(f1) ) distribute by shard(f1) to group default_group;

——— 2 insert into t1_pt (f1, f2, f3) SELECT generate_series(1000,100000) as f1, now() as f2, repeat('a',(random()*25)::integer) as order_info;

——— 3 (此步骤可以重现问题,不行就增加数量、多试几次) insert into t1_pt (f1, f2, f3) SELECT generate_series(100001,500000) as f1, now() as f2, repeat('a',(random()*25)::integer) as order_info;

Best regards, Yong

On Nov 27, 2020 11:07,q2683252notifications@github.com wrote:

表的数据分布突然从2个节点变为1个节点 这个是怎么确定出来的? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

JennyJennyChen commented 3 years ago

没看懂你的测试方法,可否直接上截图 ,然后说明你怎么看出来的数据分布变了?

softlanguage commented 3 years ago

通过sql 可以查看到只有1个节点有数据了:

SELECT xc_node_id, count(*) FROM t1_pt GROUP BY xc_node_id;
JennyJennyChen commented 3 years ago

@softlang-net 按照你的方法,测试如下: image

并没有出现你说的现象 可否把你的测试截图或者详细情况说一下

bugfyi commented 3 years ago
  1. 我使用的服务器配置: 两台4核8G
  2. generate_series单次数量加大到200万或更高,如:
    insert into t1_pt (f1, f2, f3)
    SELECT generate_series(10000,2000000) as f1,
    now() as f2, repeat('a',(random()*25)::integer) as order_info;
JennyJennyChen commented 3 years ago

单次数量加到了300W,还是没有出现 image 按照内部原理,这块应该不会出现你说的情况。你还是贴一下截图,我看一下咱们这边的操作情况

还有,关注一下你建表所属的group有几个DN: image

bugfyi commented 3 years ago

由于环境已切换为postgresxl,暂无法截图,稍晚点时间再部署一套