[Bug]: Loading data from cos with the composite primary key is 7 times slower than the single primary key.

Ariznawlll commented 5 months ago

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

Branch Name

main

Commit ID

d7db178e1c7298e2a3e4f99e7292425a7ef0ef06

Other Environment Information

- Hardware parameters:
- OS type:
- Others:

Actual Behavior

描述：单主键的表与复合主键的表load 1亿数据量，复合主键比单主键慢7倍

企业微信截图_864dc80e-a674-48db-9491-3cf1202473c3

log：http://175.178.192.213:30088/explore?panes=%7B%22AAL%22:%7B%22datasource%22:%22loki%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bnamespace%3D%5C%22branch-big-data-nightly-d7db178%5C%22%7D%20%7C%3D%20%60%60%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22loki%22%7D,%22editorMode%22:%22builder%22%7D%5D,%22range%22:%7B%22from%22:%22now-6h%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1

Expected Behavior

No response

Steps to Reproduce

Load pk 100M table schema:
create table  if not exists big_data_test.table_with_pk_for_load_100M(
id bigint auto_increment primary key,
col1 tinyint,
col2 smallint,
col3 int,
col4 bigint,
col5 tinyint unsigned,
col6 smallint unsigned,
col7 int unsigned,
col8 bigint unsigned,
col9 float,
col10 double,
col11 varchar(255),
col12 Date,
col13 DateTime,
col14 timestamp,
col15 bool,
col16 decimal(16,6),
col17 text,
col18 json,
col19 blob,
col20 binary(255),
col21 varbinary(255),
col22 vecf32(3),
col23 vecf32(3),
col24 vecf64(3),
col25 vecf64(3)
);

Load com ok index 100M table schema:
create table  if not exists big_data_test.table_with_com_pk_index_for_load_100M(
id bigint auto_increment,
col1 tinyint,
col2 smallint,
col3 int,
col4 bigint,
col5 tinyint unsigned,
col6 smallint unsigned,
col7 int unsigned,
col8 bigint unsigned,
col9 float,
col10 double,
col11 varchar(255),
col12 Date,
col13 DateTime,
col14 timestamp,
col15 bool,
col16 decimal(16,6),
col17 text,
col18 json,
col19 blob,
col20 binary(255),
col21 varbinary(255),
col22 vecf32(3),
col23 vecf32(3),
col24 vecf64(3),
col25 vecf64(3),
primary key (id, col1),
key(col3,col4),
unique key(col4)
);

trigger workflow on tke.

Additional information

No response

jensenojs commented 5 months ago

暂无进展, 后续远宁哥提了一个pr是优化相关问题的, 测试数据需要更新 https://github.com/ouyuanning/matrixone/tree/improve-insert-2nd-key

jensenojs commented 4 months ago

测试数据需要更新

jensenojs commented 4 months ago

测试数据需要更新

Ariznawlll commented 4 months ago

2024.02.26最新测试结果： commit id：ead69b441080825781ba62c0bc886f4f4b9f9ac5

企业微信截图_33473067-a677-49cb-9ede-5846c3556811

jensenojs commented 4 months ago

https://github.com/matrixorigin/matrixone/pull/14388

看来这个pr和本issue关系不大, 后面看看

暂无进展, 后续远宁哥提了一个pr是优化相关问题的, 测试数据需要更新 https://github.com/ouyuanning/matrixone/tree/improve-insert-2nd-key

jensenojs commented 3 months ago

无进展

jensenojs commented 3 months ago

无进展

jensenojs commented 3 months ago

no process

jensenojs commented 3 months ago

not working on it