feat: matched only and insert only pipeline separate

[JackTan25]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Summary about this PR

1. reduce useless pipeline build for specific workload.
2. for distributed merge into (new distributed right join strategy), if it's matched only, we don't need to accumulate useless data here (the row_number is also no need).

strategy change:

// Optimize Rule:
// for now we think right source table is small table in default.
// 1. insert only:
//      right anti join
// 2. (macthed and unmatched)
//      right outer
// 3. matched only:
//      inner join
// we will import optimizer for these join type in the future.

we will import optimizer after this pr. insert only:

merge into t1_separate as t1 using (select * from t2_separate) as t2 on t1.a = t2.a 
when not matched and t2.b = 'a5' then insert * when not matched then insert *;

## insert only semantic test
## for unmatched clause, then condition and exprs can only have source fields
merge into t1_separate as t1 using (select * from t2_separate) as t2 on t1.a = t2.a 
when not matched and t2.b = 'a5' then insert (b) values(t1.a);

matched only:

merge into t1_separate as t1 using (select * from t2_separate) as t2 on t1.a = t2.a 
when matched and t2.b = 'a5' then update * when matched then update *;

draw a conclusion: the s3 network is not stable, so we need to do much test to see the time costs and analyze the performance.And we can get this:

1. matched only can get a little performance improvement than full, the best performance of full is 9.97s in main, and our pr's matched only's performance is 8.x s.
2. insert only use right anti join, the probe phase also cost time, but we can see there is no performance decline than main. for Our test data, the source data all can matched with target table, so the main performance depends on the probe phase.and if there is no network problem, we can get 2s's best performance. So this pr is expected and we need to improve more in the future.
   • Note that: target table has 3623227415 rows, and source has 50w rows.
   • Closes #issue

     12595

   • • - This change is 

[JackTan25]

let me test this pr first.

[github-actions[bot]]

Docker Image for PR

• tag: pr-13680-3ee95c4

note: this image tag is only available for internal use, please check the internal doc for more details.

[JackTan25]

deploy@(merge_into_separate)/test_test> select count(*) from target_table_cluster;

SELECT
  count(*)
FROM
  target_table_cluster

┌────────────┐
│  count(*)  │
│   UInt64   │
├────────────┤
│ 3623227415 │
└────────────┘
1 row read in 1.809 sec. Processed 1 row, 1 B (0.55 row/s, 0 B/s)

deploy@(merge_into_separate)/test_test> select count(*) from source_table;

SELECT
  count(*)
FROM
  source_table

┌──────────┐
│ count(*) │
│  UInt64  │
├──────────┤
│   500000 │
└──────────┘
1 row read in 0.802 sec. Processed 1 row, 1 B (1.25 row/s, 1 B/s)

deploy@(merge_into_separate)/test_test> set enable_experimental_merge_into = 1;

SET
  enable_experimental_merge_into = 1

0 row read in 1.290 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(merge_into_separate)/test_test> set enable_distributed_merge_into = 0;

SET
  enable_distributed_merge_into = 0

0 row read in 0.611 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(merge_into_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when
matched  then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 12.878 sec. Processed 1.5 million row, 182.15 MiB (116.48 thousand row/s, 14.14 MiB/s)

deploy@(merge_into_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 0.805 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

## matched only test
deploy@(merge_into_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when
matched  then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 9.997 sec. Processed 1.5 million row, 182.15 MiB (150.05 thousand row/s, 18.22 MiB/s)

deploy@(merge_into_separate)/test_test> select count(*) from target_table_cluster;

SELECT
  count(*)
FROM
  target_table_cluster

┌────────────┐
│  count(*)  │
│   UInt64   │
├────────────┤
│ 3623227415 │
└────────────┘
1 row read in 1.460 sec. Processed 1 row, 1 B (0.68 row/s, 0 B/s)

deploy@(merge_into_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when
matched  then update *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

0 row read in 9.376 sec. Processed 1.5 million row, 182.15 MiB (159.99 thousand row/s, 19.43 MiB/s)

deploy@(merge_into_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

## insert only
MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 21.488 sec. Processed 4.06 million row, 552.53 MiB (189.01 thousand row/s, 25.71 MiB/s)

deploy@(merge_into_separate)/test_test> select count(*) from target_table_cluster;

SELECT
  count(*)
FROM
  target_table_cluster

┌────────────┐
│  count(*)  │
│   UInt64   │
├────────────┤
│ 3623227415 │
└────────────┘
1 row read in 1.669 sec. Processed 1 row, 1 B (0.60 row/s, 0 B/s)

[JackTan25]

We pass the basic correctness on cloud test. We can see matched only 's improvment from 9.997 to 9.376s .But it seems the insert only is very slow. let's change into draft.

[JackTan25]

in standalone mode, matched only is slower than distributed mode expectedly, but insert only in standalone mode is same with distributed mode.

deploy@(merge_into_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 22.052 sec. Processed 4.26 million row, 600.08 MiB (193.09 thousand row/s, 27.21 MiB/s)

deploy@(merge_into_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

0 row read in 32.359 sec. Processed 1.5 million row, 182.15 MiB (46.36 thousand row/s, 5.63 MiB/s)

deploy@(merge_into_separate)/test_test> select count(*) from target_table_cluster;

SELECT
  count(*)
FROM
  target_table_cluster

┌────────────┐
│  count(*)  │
│   UInt64   │
├────────────┤
│ 3623227415 │
└────────────┘
1 row read in 0.753 sec. Processed 1 row, 1 B (1.33 row/s, 1 B/s)

[github-actions[bot]]

Docker Image for PR

• tag: pr-13680-e20bd9c

note: this image tag is only available for internal use, please check the internal doc for more details.

[JackTan25]

deploy@(merge_into_separate)/test_test> truncate table system.metrics;

TRUNCATE TABLE system.metrics

0 row read in 6.505 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(merge_into_separate)/test_test> truncate table system.metrics;

TRUNCATE TABLE system.metrics

0 row read in 0.704 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(merge_into_separate)/test_test> select count(*) from fuse_segment('test_test','target_table_cluster');

SELECT
  count(*)
FROM
  fuse_segment('test_test', 'target_table_cluster')

┌──────────┐
│ count(*) │
│  UInt64  │
├──────────┤
│        7 │
└──────────┘
1 row read in 0.990 sec. Processed 7 row, 757 B (7.07 row/s, 764 B/s)

deploy@(merge_into_separate)/test_test> set enable_distributed_merge_into =1;merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update *;

SET
  enable_distributed_merge_into = 1

0 row read in 0.630 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

0 row read in 14.208 sec. Processed 1.5 million row, 182.15 MiB (105.57 thousand row/s, 12.82 MiB/s)

deploy@(merge_into_separate)/test_test> select * from system.metrics where metric like '%prun%' order by node;

SELECT
  *
FROM
  system.metrics
WHERE
  metric LIKE '%prun%'
ORDER BY
  node

┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│          node          │                     metric                    │    kind   │ labels │                                                                                                                                                                                                                                                                        value                                                                                                                                                                                                                                                                        │
│         String         │                     String                    │   String  │ String │                                                                                                                                                                                                                                                                        String                                                                                                                                                                                                                                                                       │
├────────────────────────┼───────────────────────────────────────────────┼───────────┼────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_segments_range_pruning_before_total      │ untyped   │ {}     │ 16.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_bytes_segment_range_pruning_before_total │ untyped   │ {}     │ 1409973640904.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_segments_range_pruning_after_total       │ untyped   │ {}     │ 10.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_bytes_segment_range_pruning_after_total  │ untyped   │ {}     │ 705157820596.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_blocks_range_pruning_before_total        │ untyped   │ {}     │ 3.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_bytes_block_bloom_pruning_before_total   │ untyped   │ {}     │ 255500248.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_blocks_range_pruning_after_total         │ untyped   │ {}     │ 3.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_bytes_block_bloom_pruning_after_total    │ untyped   │ {}     │ 255500248.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_pruning_milliseconds_sum                 │ untyped   │ {}     │ 0.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_pruning_milliseconds_count               │ untyped   │ {}     │ 3.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 │
│ l2NQ6VEOavrdohc9ViKyh1 │ fuse_pruning_milliseconds                     │ histogram │ {}     │ [{"less_than":10.0,"count":3.0},{"less_than":50.0,"count":3.0},{"less_than":100.0,"count":3.0},{"less_than":250.0,"count":3.0},{"less_than":500.0,"count":3.0},{"less_than":1000.0,"count":3.0},{"less_than":2500.0,"count":3.0},{"less_than":5000.0,"count":3.0},{"less_than":10000.0,"count":3.0},{"less_than":20000.0,"count":3.0},{"less_than":30000.0,"count":3.0},{"less_than":60000.0,"count":3.0},{"less_than":300000.0,"count":3.0},{"less_than":600000.0,"count":3.0},{"less_than":1800000.0,"count":3.0},{"less_than":null,"count":3.0}] │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
11 rows read in 1.010 sec. Processed 454 rows, 93.62 KiB (449.39 rows/s, 92.67 KiB/s)

seems the pruning metrics have problem. We have only 7 segments, but we get 16, cc @dantengsky

[JackTan25]

This is tested at v1.2.212-nightly

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 2.499 sec. Processed 1.5 million row, 116.83 MiB (600.29 thousand row/s, 46.75 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 2.603 sec. Processed 1.5 million row, 116.83 MiB (576.17 thousand row/s, 44.87 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *
0 row read in 21.011 sec. Processed 3.13 million row, 393.89 MiB (148.96 thousand row/s, 18.75 MiB/s)

deploy@(large_merge_separate_main)/test_test> select count(*) from target_table_cluster;

SELECT
  count(*)
FROM
  target_table_cluster

┌────────────┐
│  count(*)  │
│   UInt64   │
├────────────┤
│ 3623227415 │
└────────────┘
1 row read in 4.675 sec. Processed 1 row, 1 B (0.21 row/s, 0 B/s)

The unstable test performance should be related to the recluster and compact effect. But all in all, the highest time spent is below 23s in distributed mod for all tests.

[JackTan25]

seems the build_static_filter will do the pruning once, and then the merge into execution will do it anain.

[JackTan25]

It should be the unstable network cause the test result.

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 13.054 sec. Processed 1.5 million row, 182.15 MiB (114.9 thousand row/s, 13.95 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 34.849 sec. Processed 3.13 million row, 460.21 MiB (89.7 thousand row/s, 13.21 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 9.568 sec. Processed 1.5 million row, 182.15 MiB (156.77 thousand row/s, 19.04 MiB/s)

[JackTan25]

After test again, we find out that:The best matched only can get 8.x s, and full operation (matched and unmatched) can get about 10s. But as to the network problem, sometimes the read is slow and cause the total time is unstable.

deploy@(large_merge_separate_main)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 0.825 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 31.687 sec. Processed 3.15 million row, 464.32 MiB (99.44 thousand row/s, 14.65 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 13.123 sec. Processed 1.5 million row, 182.15 MiB (114.31 thousand row/s, 13.88 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 16.169 sec. Processed 1.5 million row, 182.15 MiB (92.77 thousand row/s, 11.27 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 11.310 sec. Processed 1.5 million row, 182.15 MiB (132.63 thousand row/s, 16.11 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 10.604 sec. Processed 1.5 million row, 182.15 MiB (141.45 thousand row/s, 17.18 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 11.379 sec. Processed 1.5 million row, 182.15 MiB (131.82 thousand row/s, 16.01 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 10.800 sec. Processed 1.5 million row, 182.15 MiB (138.89 thousand row/s, 16.87 MiB/s)

deploy@(large_merge_separate_main)/test_test> show warehouses;

SHOW warehouses

┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│            name           │   state   │  size  │      version     │ auto_suspend │    cache_size   │      created_on     │
│           String          │   String  │ String │      String      │    UInt32    │ Nullable(Int64) │      Timestamp      │
├───────────────────────────┼───────────┼────────┼──────────────────┼──────────────┼─────────────────┼─────────────────────┤
│ zhihanz                   │ Suspended │ Medium │                  │           60 │            NULL │ 2022-11-24 05:25:58 │
│ large-hbau                │ Suspended │ Large  │                  │           60 │            NULL │ 2023-06-06 04:55:44 │
│ xsmall-8vyb               │ Suspended │ XSmall │                  │         1800 │            NULL │ 2023-08-22 06:35:33 │
│ small-2hus                │ Suspended │ Small  │                  │          600 │            NULL │ 2023-08-29 10:05:32 │
│ xsmall-q77r               │ Suspended │ XSmall │                  │           60 │            NULL │ 2023-09-21 11:08:42 │
│ pctest                    │ Suspended │ XSmall │                  │          600 │            NULL │ 2023-10-09 08:23:42 │
│ system                    │ Running   │ XSmall │                  │         3600 │            NULL │ 2023-10-12 13:25:23 │
│ reload                    │ Suspended │ Small  │                  │           60 │            NULL │ 2023-10-18 08:31:27 │
│ test-filter               │ Suspended │ XLarge │ pr-13511-7a00710 │           60 │              25 │ 2023-11-01 13:49:57 │
│ yazhou                    │ Suspended │ Small  │                  │           60 │            NULL │ 2023-11-06 10:51:04 │
│ yazhou2                   │ Suspended │ XSmall │                  │            0 │            NULL │ 2023-11-08 07:05:16 │
│ bh-pr13548                │ Suspended │ XSmall │ pr-13548-308de57 │           60 │               0 │ 2023-11-16 02:46:18 │
│ merge_into_separate       │ Suspended │ XLarge │ pr-13680-e20bd9c │           60 │              25 │ 2023-11-25 05:10:00 │
│ large_merge_separate      │ Suspended │ Large  │ pr-13680-e20bd9c │           60 │              25 │ 2023-11-25 06:38:46 │
│ large_merge_separate_main │ Running   │ Large  │ v1.2.212-nightly │           60 │              25 │ 2023-11-25 06:48:33 │
│ large_main_with_stream    │ Suspended │ Large  │ pr-13806-14e2817 │           60 │              25 │ 2023-11-25 11:14:31 │
│ large_main_without_stream │ Suspended │ Large  │ pr-13805-40d402d │           60 │              25 │ 2023-11-25 11:14:57 │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
deploy@(large_merge_separate_main)/test_test> show warehouses;

SHOW warehouses

┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│            name           │   state   │  size  │      version     │ auto_suspend │    cache_size   │      created_on     │
│           String          │   String  │ String │      String      │    UInt32    │ Nullable(Int64) │      Timestamp      │
├───────────────────────────┼───────────┼────────┼──────────────────┼──────────────┼─────────────────┼─────────────────────┤
│ zhihanz                   │ Suspended │ Medium │                  │           60 │            NULL │ 2022-11-24 05:25:58 │
│ large-hbau                │ Suspended │ Large  │                  │           60 │            NULL │ 2023-06-06 04:55:44 │
│ xsmall-8vyb               │ Suspended │ XSmall │                  │         1800 │            NULL │ 2023-08-22 06:35:33 │
│ small-2hus                │ Suspended │ Small  │                  │          600 │            NULL │ 2023-08-29 10:05:32 │
│ xsmall-q77r               │ Suspended │ XSmall │                  │           60 │            NULL │ 2023-09-21 11:08:42 │
│ pctest                    │ Suspended │ XSmall │                  │          600 │            NULL │ 2023-10-09 08:23:42 │
│ system                    │ Running   │ XSmall │                  │         3600 │            NULL │ 2023-10-12 13:25:23 │
│ reload                    │ Suspended │ Small  │                  │           60 │            NULL │ 2023-10-18 08:31:27 │
│ test-filter               │ Suspended │ XLarge │ pr-13511-7a00710 │           60 │              25 │ 2023-11-01 13:49:57 │
│ yazhou                    │ Suspended │ Small  │                  │           60 │            NULL │ 2023-11-06 10:51:04 │
│ yazhou2                   │ Suspended │ XSmall │                  │            0 │            NULL │ 2023-11-08 07:05:16 │
│ bh-pr13548                │ Suspended │ XSmall │ pr-13548-308de57 │           60 │               0 │ 2023-11-16 02:46:18 │
│ merge_into_separate       │ Suspended │ XLarge │ pr-13680-e20bd9c │           60 │              25 │ 2023-11-25 05:10:00 │
│ large_merge_separate      │ Suspended │ Large  │ pr-13680-e20bd9c │           60 │              25 │ 2023-11-25 06:38:46 │
│ large_merge_separate_main │ Suspended │ Large  │ v1.2.212-nightly │           60 │              25 │ 2023-11-25 06:48:33 │
│ large_main_with_stream    │ Suspended │ Large  │ pr-13806-14e2817 │           60 │              25 │ 2023-11-25 11:14:31 │
│ large_main_without_stream │ Suspended │ Large  │ pr-13805-40d402d │           60 │              25 │ 2023-11-25 11:14:57 │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
deploy@(large_merge_separate_main)/test_test> use warehouse large_merge_separate;

USE warehouse large_merge_separate

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 5.131 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 0.740 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 13.054 sec. Processed 1.5 million row, 182.15 MiB (114.9 thousand row/s, 13.95 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 34.849 sec. Processed 3.13 million row, 460.21 MiB (89.7 thousand row/s, 13.21 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 9.568 sec. Processed 1.5 million row, 182.15 MiB (156.77 thousand row/s, 19.04 MiB/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 7.138 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 1.002 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> select * from system.clusters;

SELECT
  *
FROM
  system.clusters

┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│          name          │     host    │  port  │                                     version                                     │
│         String         │    String   │ UInt16 │                                      String                                     │
├────────────────────────┼─────────────┼────────┼─────────────────────────────────────────────────────────────────────────────────┤
│ 6wRohUQag83YrsMcXAl7A7 │ 10.1.63.198 │   9191 │ v1.2.223-nightly-e20bd9c2bc(rust-1.75.0-nightly-2023-11-24T12:31:32.416197228Z) │
│ S9DxYbESPi72o3cj5gnDo1 │ 10.1.13.223 │   9191 │ v1.2.223-nightly-e20bd9c2bc(rust-1.75.0-nightly-2023-11-24T12:31:32.416197228Z) │
│ cqz9TrjRT0OwQ1jmw3tre7 │ 10.1.7.61   │   9191 │ v1.2.223-nightly-e20bd9c2bc(rust-1.75.0-nightly-2023-11-24T12:31:32.416197228Z) │
│ v4t1wmRWvMvU47ZPLyxmq5 │ 10.1.19.200 │   9191 │ v1.2.223-nightly-e20bd9c2bc(rust-1.75.0-nightly-2023-11-24T12:31:32.416197228Z) │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
4 rows read in 0.692 sec. Processed 4 rows, 574 B (5.78 rows/s, 829 B/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 5.049 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> select * from system.clusters;

SELECT
  *
FROM
  system.clusters

┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│          name          │     host    │  port  │                                     version                                     │
│         String         │    String   │ UInt16 │                                      String                                     │
├────────────────────────┼─────────────┼────────┼─────────────────────────────────────────────────────────────────────────────────┤
│ 6wRohUQag83YrsMcXAl7A7 │ 10.1.63.198 │   9191 │ v1.2.223-nightly-e20bd9c2bc(rust-1.75.0-nightly-2023-11-24T12:31:32.416197228Z) │
│ S9DxYbESPi72o3cj5gnDo1 │ 10.1.13.223 │   9191 │ v1.2.223-nightly-e20bd9c2bc(rust-1.75.0-nightly-2023-11-24T12:31:32.416197228Z) │
│ cqz9TrjRT0OwQ1jmw3tre7 │ 10.1.7.61   │   9191 │ v1.2.223-nightly-e20bd9c2bc(rust-1.75.0-nightly-2023-11-24T12:31:32.416197228Z) │
│ v4t1wmRWvMvU47ZPLyxmq5 │ 10.1.19.200 │   9191 │ v1.2.223-nightly-e20bd9c2bc(rust-1.75.0-nightly-2023-11-24T12:31:32.416197228Z) │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
4 rows read in 1.023 sec. Processed 4 rows, 574 B (3.91 rows/s, 561 B/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 1.091 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 34.367 sec. Processed 3.04 million row, 444.35 MiB (88.45 thousand row/s, 12.93 MiB/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 0.946 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 12.458 sec. Processed 1.5 million row, 182.15 MiB (120.41 thousand row/s, 14.62 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 13.150 sec. Processed 1.5 million row, 182.15 MiB (114.06 thousand row/s, 13.85 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update * when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *
  WHEN NOT matched THEN
INSERT
  *

0 row read in 10.922 sec. Processed 1.5 million row, 182.15 MiB (137.34 thousand row/s, 16.68 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

0 row read in 28.040 sec. Processed 3.15 million row, 472.56 MiB (112.29 thousand row/s, 16.85 MiB/s)

deploy@(large_merge_separate)/test_test> explain merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update *;

EXPLAIN MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

-[ EXPLAIN ]-----------------------------------
Merge Into { catalog: "default", database: "test_test", table: "target_table_cluster", table_id: 5877621, join: SExpr { plan: Exchange(Merge), children: [SExpr { plan: Join(Join { left_conditions: [BoundColumnRef(BoundColumnRef { span: Some(90..102), column: ColumnBinding { database_name: None, table_name: Some("t1"), column_position: Some(2), table_index: Some(1), column_name: "l_partkey", index: 17, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), BoundColumnRef(BoundColumnRef { span: Some(122..135), column: ColumnBinding { database_name: None, table_name: Some("t1"), column_position: Some(1), table_index: Some(1), column_name: "l_orderkey", index: 16, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), BoundColumnRef(BoundColumnRef { span: Some(156..168), column: ColumnBinding { database_name: None, table_name: Some("t1"), column_position: Some(3), table_index: Some(1), column_name: "l_suppkey", index: 18, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), BoundColumnRef(BoundColumnRef { span: Some(188..203), column: ColumnBinding { database_name: None, table_name: Some("t1"), column_position: Some(4), table_index: Some(1), column_name: "l_linenumber", index: 19, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), BoundColumnRef(BoundColumnRef { span: Some(226..239), column: ColumnBinding { database_name: None, table_name: Some("t1"), column_position: Some(11), table_index: Some(1), column_name: "l_shipdate", index: 26, data_type: Date, visibility: Visible, virtual_computed_expr: None } })], right_conditions: [BoundColumnRef(BoundColumnRef { span: Some(105..117), column: ColumnBinding { database_name: None, table_name: Some("t2"), column_position: Some(2), table_index: Some(0), column_name: "l_partkey", index: 1, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), BoundColumnRef(BoundColumnRef { span: Some(138..151), column: ColumnBinding { database_name: None, table_name: Some("t2"), column_position: Some(1), table_index: Some(0), column_name: "l_orderkey", index: 0, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), BoundColumnRef(BoundColumnRef { span: Some(171..183), column: ColumnBinding { database_name: None, table_name: Some("t2"), column_position: Some(3), table_index: Some(0), column_name: "l_suppkey", index: 2, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), BoundColumnRef(BoundColumnRef { span: Some(206..221), column: ColumnBinding { database_name: None, table_name: Some("t2"), column_position: Some(4), table_index: Some(0), column_name: "l_linenumber", index: 3, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), BoundColumnRef(BoundColumnRef { span: Some(242..255), column: ColumnBinding { database_name: None, table_name: Some("t2"), column_position: Some(11), table_index: Some(0), column_name: "l_shipdate", index: 10, data_type: Date, visibility: Visible, virtual_computed_expr: None } })], non_equi_conditions: [], join_type: Inner, marker_index: None, from_correlated_subquery: false, contain_runtime_filter: false, need_hold_hash_table: true }), children: [SExpr { plan: Scan(Scan { table_index: 1, columns: {27, 28, 17, 32, 20, 16, 25, 19, 22, 18, 23, 26, 21, 31, 24, 29, 30}, push_down_predicates: None, limit: None, order_by: None, prewhere: None, agg_index: None, statistics: Statistics { statistics: Some(TableStatistics { num_rows: Some(3623227415), data_size: Some(704902320348), data_size_compressed: Some(99557345679), index_size: Some(12004869172), number_of_blocks: Some(6266), number_of_segments: Some(7) }), col_stats: {31: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 122, 122, 115])), ndv: Some(3623227415), null_count: 0 }), 23: None, 30: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 122, 119, 90])), ndv: Some(3623227415), null_count: 0 }), 18: Some(BasicColumnStatistics { min: Some(Int(-9223371363634053650)), max: Some(Int(9223368941606612118)), ndv: Some(3623227415), null_count: 0 }), 20: None, 16: Some(BasicColumnStatistics { min: Some(Int(-9223324117790460131)), max: Some(Int(9223365100798332574)), ndv: Some(3623227415), null_count: 0 }), 25: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 122, 122, 119])), ndv: Some(3623227415), null_count: 0 }), 19: Some(BasicColumnStatistics { min: Some(Int(-9223366450700995069)), max: Some(Int(9223364076393176495)), ndv: Some(3623227415), null_count: 0 }), 28: Some(BasicColumnStatistics { min: Some(Int(-354285)), max: Some(Int(2932896)), ndv: Some(3287182), null_count: 0 }), 24: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 122, 121, 81])), ndv: Some(3623227415), null_count: 0 }), 27: Some(BasicColumnStatistics { min: Some(Int(-354285)), max: Some(Int(2932896)), ndv: Some(3287182), null_count: 0 }), 22: None, 29: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 122, 119, 71])), ndv: Some(3623227415), null_count: 0 }), 26: Some(BasicColumnStatistics { min: Some(Int(0)), max: Some(Int(2932888)), ndv: Some(2932889), null_count: 0 }), 21: None, 17: Some(BasicColumnStatistics { min: Some(Int(-9223371794626041650)), max: Some(Int(9223367531909014139)), ndv: Some(3623227415), null_count: 0 })} } }), children: [], original_group: None, rel_prop: Mutex { data: None, poisoned: false, .. }, stat_info: Mutex { data: None, poisoned: false, .. }, applied_rules: AppliedRules { rules: RuleSet { rules: RoaringBitmap<[]> } } }, SExpr { plan: Exchange(Broadcast), children: [SExpr { plan: AddRowNumber(AddRowNumber), children: [SExpr { plan: EvalScalar(EvalScalar { items: [ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(1), table_index: Some(0), column_name: "l_orderkey", index: 0, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), index: 0 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(2), table_index: Some(0), column_name: "l_partkey", index: 1, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), index: 1 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(3), table_index: Some(0), column_name: "l_suppkey", index: 2, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), index: 2 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(4), table_index: Some(0), column_name: "l_linenumber", index: 3, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), index: 3 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(5), table_index: Some(0), column_name: "l_quantity", index: 4, data_type: Decimal(Decimal128(DecimalSize { precision: 15, scale: 2 })), visibility: Visible, virtual_computed_expr: None } }), index: 4 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(6), table_index: Some(0), column_name: "l_extendedprice", index: 5, data_type: Decimal(Decimal128(DecimalSize { precision: 15, scale: 2 })), visibility: Visible, virtual_computed_expr: None } }), index: 5 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(7), table_index: Some(0), column_name: "l_discount", index: 6, data_type: Decimal(Decimal128(DecimalSize { precision: 15, scale: 2 })), visibility: Visible, virtual_computed_expr: None } }), index: 6 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(8), table_index: Some(0), column_name: "l_tax", index: 7, data_type: Decimal(Decimal128(DecimalSize { precision: 15, scale: 2 })), visibility: Visible, virtual_computed_expr: None } }), index: 7 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(9), table_index: Some(0), column_name: "l_returnflag", index: 8, data_type: String, visibility: Visible, virtual_computed_expr: None } }), index: 8 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(10), table_index: Some(0), column_name: "l_linestatus", index: 9, data_type: String, visibility: Visible, virtual_computed_expr: None } }), index: 9 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(11), table_index: Some(0), column_name: "l_shipdate", index: 10, data_type: Date, visibility: Visible, virtual_computed_expr: None } }), index: 10 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(12), table_index: Some(0), column_name: "l_commitdate", index: 11, data_type: Date, visibility: Visible, virtual_computed_expr: None } }), index: 11 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(13), table_index: Some(0), column_name: "l_receiptdate", index: 12, data_type: Date, visibility: Visible, virtual_computed_expr: None } }), index: 12 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(14), table_index: Some(0), column_name: "l_shipinstruct", index: 13, data_type: String, visibility: Visible, virtual_computed_expr: None } }), index: 13 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(15), table_index: Some(0), column_name: "l_shipmode", index: 14, data_type: String, visibility: Visible, virtual_computed_expr: None } }), index: 14 }, ScalarItem { scalar: BoundColumnRef(BoundColumnRef { span: Some(60..61), column: ColumnBinding { database_name: Some("test_test"), table_name: Some("source_table"), column_position: Some(16), table_index: Some(0), column_name: "l_comment", index: 15, data_type: String, visibility: Visible, virtual_computed_expr: None } }), index: 15 }] }), children: [SExpr { plan: Scan(Scan { table_index: 0, columns: {9, 1, 0, 10, 3, 5, 2, 4, 12, 8, 13, 6, 15, 11, 7, 14}, push_down_predicates: None, limit: None, order_by: None, prewhere: None, agg_index: None, statistics: Statistics { statistics: Some(TableStatistics { num_rows: Some(500000), data_size: Some(84500104), data_size_compressed: Some(36220894), index_size: Some(6677810), number_of_blocks: Some(1), number_of_segments: Some(1) }), col_stats: {6: None, 8: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 122, 109, 83])), ndv: Some(500000), null_count: 0 }), 5: None, 15: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 122, 83, 73])), ndv: Some(500000), null_count: 0 }), 10: Some(BasicColumnStatistics { min: Some(Int(-354274)), max: Some(Int(2932885)), ndv: Some(500000), null_count: 0 }), 2: Some(BasicColumnStatistics { min: Some(Int(-9223370003562995123)), max: Some(Int(9223362884844597891)), ndv: Some(500000), null_count: 0 }), 7: None, 9: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 121, 87, 99])), ndv: Some(500000), null_count: 0 }), 0: Some(BasicColumnStatistics { min: Some(Int(-9223324117790460131)), max: Some(Int(9223365100798332573)), ndv: Some(500000), null_count: 0 }), 13: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 122, 115, 83])), ndv: Some(500000), null_count: 0 }), 4: None, 11: Some(BasicColumnStatistics { min: Some(Int(-354283)), max: Some(Int(2932885)), ndv: Some(500000), null_count: 0 }), 14: Some(BasicColumnStatistics { min: Some(Bytes([])), max: Some(Bytes([122, 122, 122, 100, 121])), ndv: Some(500000), null_count: 0 }), 3: Some(BasicColumnStatistics { min: Some(Int(-9223319855629578086)), max: Some(Int(9223319913925052051)), ndv: Some(500000), null_count: 0 }), 1: Some(BasicColumnStatistics { min: Some(Int(-9223334733738144154)), max: Some(Int(9223363371178483560)), ndv: Some(500000), null_count: 0 }), 12: Some(BasicColumnStatistics { min: Some(Int(-354266)), max: Some(Int(2932893)), ndv: Some(500000), null_count: 0 })} } }), children: [], original_group: None, rel_prop: Mutex { data: None, poisoned: false, .. }, stat_info: Mutex { data: None, poisoned: false, .. }, applied_rules: AppliedRules { rules: RuleSet { rules: RoaringBitmap<[]> } } }], original_group: None, rel_prop: Mutex { data: None, poisoned: false, .. }, stat_info: Mutex { data: None, poisoned: false, .. }, applied_rules: AppliedRules { rules: RuleSet { rules: RoaringBitmap<[]> } } }], original_group: None, rel_prop: Mutex { data: None, poisoned: false, .. }, stat_info: Mutex { data: None, poisoned: false, .. }, applied_rules: AppliedRules { rules: RuleSet { rules: RoaringBitmap<[]> } } }], original_group: None, rel_prop: Mutex { data: None, poisoned: false, .. }, stat_info: Mutex { data: None, poisoned: false, .. }, applied_rules: AppliedRules { rules: RuleSet { rules: RoaringBitmap<[]> } } }], original_group: None, rel_prop: Mutex { data: None, poisoned: false, .. }, stat_info: Mutex { data: None, poisoned: false, .. }, applied_rules: AppliedRules { rules: RuleSet { rules: RoaringBitmap<[]> } } }], original_group: None, rel_prop: Mutex { data: None, poisoned: false, .. }, stat_info: Mutex { data: None, poisoned: false, .. }, applied_rules: AppliedRules { rules: RuleSet { rules: RoaringBitmap<[]> } } }, matched: [MatchedEvaluator { condition: None, update: Some({3: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_linenumber", index: 3, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), 7: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_tax", index: 7, data_type: Decimal(Decimal128(DecimalSize { precision: 15, scale: 2 })), visibility: Visible, virtual_computed_expr: None } }), 9: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_linestatus", index: 9, data_type: String, visibility: Visible, virtual_computed_expr: None } }), 6: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_discount", index: 6, data_type: Decimal(Decimal128(DecimalSize { precision: 15, scale: 2 })), visibility: Visible, virtual_computed_expr: None } }), 10: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_shipdate", index: 10, data_type: Date, visibility: Visible, virtual_computed_expr: None } }), 1: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_partkey", index: 1, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), 13: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_shipinstruct", index: 13, data_type: String, visibility: Visible, virtual_computed_expr: None } }), 0: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_orderkey", index: 0, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), 12: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_receiptdate", index: 12, data_type: Date, visibility: Visible, virtual_computed_expr: None } }), 15: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_comment", index: 15, data_type: String, visibility: Visible, virtual_computed_expr: None } }), 5: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_extendedprice", index: 5, data_type: Decimal(Decimal128(DecimalSize { precision: 15, scale: 2 })), visibility: Visible, virtual_computed_expr: None } }), 4: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_quantity", index: 4, data_type: Decimal(Decimal128(DecimalSize { precision: 15, scale: 2 })), visibility: Visible, virtual_computed_expr: None } }), 11: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_commitdate", index: 11, data_type: Date, visibility: Visible, virtual_computed_expr: None } }), 2: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_suppkey", index: 2, data_type: Number(Int64), visibility: Visible, virtual_computed_expr: None } }), 14: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_shipmode", index: 14, data_type: String, visibility: Visible, virtual_computed_expr: None } }), 8: BoundColumnRef(BoundColumnRef { span: None, column: ColumnBinding { database_name: None, table_name: None, column_position: None, table_index: None, column_name: "l_returnflag", index: 8, data_type: String, visibility: Visible, virtual_computed_expr: None } })}) }], unmateched: [], distributed: true }

1 row explain in 0.942 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

0 row read in 28.975 sec. Processed 2.89 million row, 416.51 MiB (99.77 thousand row/s, 14.37 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

0 row read in 8.931 sec. Processed 1.5 million row, 182.15 MiB (167.95 thousand row/s, 20.40 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

0 row read in 28.170 sec. Processed 3.19 million row, 474.37 MiB (113.31 thousand row/s, 16.84 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

0 row read in 32.228 sec. Processed 3.28 million row, 489.69 MiB (101.84 thousand row/s, 15.19 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when matched then update *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN matched THEN
UPDATE
  *

0 row read in 10.689 sec. Processed 1.5 million row, 182.15 MiB (140.33 thousand row/s, 17.04 MiB/s)

[JackTan25]

And insert only is same with main repo without performance decline(about 21s) (sometimes the main costs only 2s,but in most time, it costs 20+s, it should be the network problem):

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 21.588 sec. Processed 2.83 million row, 340.38 MiB (130.87 thousand row/s, 15.77 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 22.050 sec. Processed 2.74 million row, 324.47 MiB (124.21 thousand row/s, 14.72 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 20.393 sec. Processed 3.06 million row, 390.09 MiB (150.27 thousand row/s, 19.13 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 20.549 sec. Processed 3.04 million row, 386.22 MiB (147.98 thousand row/s, 18.79 MiB/s)

main test:

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 21.723 sec. Processed 2.85 million row, 352.18 MiB (131.08 thousand row/s, 16.21 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 6.027 sec. Processed 1.5 million row, 116.83 MiB (248.89 thousand row/s, 19.38 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 2.559 sec. Processed 1.5 million row, 116.83 MiB (586.12 thousand row/s, 45.65 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 20.841 sec. Processed 2.85 million row, 353.46 MiB (136.51 thousand row/s, 16.96 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 21.889 sec. Processed 2.85 million row, 340.98 MiB (130.14 thousand row/s, 15.58 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 22.023 sec. Processed 3.26 million row, 416.62 MiB (147.92 thousand row/s, 18.92 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 3.460 sec. Processed 1.5 million row, 116.83 MiB (433.52 thousand row/s, 33.76 MiB/s)

deploy@(large_merge_separate_main)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 20.791 sec. Processed 3.15 million row, 398.95 MiB (151.54 thousand row/s, 19.19 MiB/s)

deploy@(large_merge_separate_main)/test_test> select count(*) from   target_table_cluster;

SELECT
  count(*)
FROM
  target_table_cluster

┌────────────┐
│  count(*)  │
│   UInt64   │
├────────────┤
│ 3623227415 │
└────────────┘
1 row read in 0.753 sec. Processed 1 row, 1 B (1.33 row/s, 1 B/s)

[JackTan25]

And insert only is same with main repo without performance decline(about 21s) (sometimes the main costs only 2s,but in most time, it costs 20+s, it should be the network problem):

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 21.588 sec. Processed 2.83 million row, 340.38 MiB (130.87 thousand row/s, 15.77 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 22.050 sec. Processed 2.74 million row, 324.47 MiB (124.21 thousand row/s, 14.72 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 20.393 sec. Processed 3.06 million row, 390.09 MiB (150.27 thousand row/s, 19.13 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

test again, we can also get the best 2s in this pr:

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 2.495 sec. Processed 1.5 million row, 116.83 MiB (601.09 thousand row/s, 46.82 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

⠂ [00:00:08] Processing 2.74 million/3.62 billion (337.3 thousand rows/s), 328.86 MiB/92.77 GiB (40.55 MiB/s) ░                     (2318.7s)
0 row read in 20.959 sec. Processed 2.74 million row, 328.86 MiB (130.52 thousand row/s, 15.69 MiB/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 20.815 sec. Processed 3.04 million row, 385.11 MiB (146.25 thousand row/s, 18.50 MiB/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 1.297 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 0.677 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 0.654 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 20.991 sec. Processed 3.06 million row, 383.08 MiB (145.98 thousand row/s, 18.25 MiB/s)

deploy@(large_merge_separate)/test_test> set enable_distributed_merge_into = 1;

SET
  enable_distributed_merge_into = 1

0 row read in 0.667 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

deploy@(large_merge_separate)/test_test> merge into target_table_cluster as t1 using (select * from source_table) as t2
on t1.l_partkey = t2.l_partkey and t1.l_orderkey = t2.l_orderkey and
t1.l_suppkey = t2.l_suppkey and t1.l_linenumber = t2.l_linenumber and t1.l_shipdate = t2.l_shipdate when not matched then insert *;

MERGE INTO target_table_cluster AS t1 USING (
  SELECT
    *
  FROM
    source_table
) AS t2 ON t1.l_partkey = t2.l_partkey
AND t1.l_orderkey = t2.l_orderkey
AND t1.l_suppkey = t2.l_suppkey
AND t1.l_linenumber = t2.l_linenumber
AND t1.l_shipdate = t2.l_shipdate
WHEN NOT matched THEN
INSERT
  *

0 row read in 2.385 sec. Processed 1.5 million row, 116.83 MiB (628.81 thousand row/s, 48.97 MiB/s)

[JackTan25]

for now, we can't run test_scripts. cc @zhyass @dantengsky lock bug.

[2023-11-25T13:02:40Z INFO  test_replace_recluster] executing table maintenance batch : 11039
[2023-11-25T13:02:40Z INFO  test_replace_recluster] Err. maintenance batch : 11039. APIError: ResponseError with 2015: table 'test_order' is locked, please retry compaction later
[2023-11-25T13:02:40Z INFO  test_replace_recluster] Err. maintenance batch : 11039. APIError: ResponseError with 2015: table 'test_order' is locked, please retry compaction later
[2023-11-25T13:02:40Z INFO  test_replace_recluster] Ok. maintenance batch : 11039
[2023-11-25T13:02:40Z INFO  test_replace_recluster] Err. maintenance batch : 11039. APIError: ResponseError with 2015: table 'test_order' is locked, please retry recluster later
[2023-11-25T13:02:40Z INFO  test_replace_recluster] executing table maintenance batch : 11040
[2023-11-25T13:02:40Z INFO  test_replace_recluster] Err. maintenance batch : 11040. APIError: ResponseError with 2015: table 'test_order' is locked, please retry compaction later
[2023-11-25T13:02:40Z INFO  test_replace_recluster] Err. maintenance batch : 11040. APIError: ResponseError with 2015: table 'test_order' is locked, please retry compaction later
[2023-11-25T13:02:40Z INFO  test_replace_recluster] Ok. maintenance batch : 11040
[2023-11-25T13:02:40Z INFO  test_replace_recluster] Err. maintenance batch : 11040. APIError: ResponseError with 2015: table 'test_order' is locked, please retry recluster later
[2023-11-25T13:02:40Z INFO  test_replace_recluster] executing table maintenance batch : 11041
[2023-11-25T13:02:40Z INFO  test_replace_recluster] Err. maintenance batch : 11041. APIError: ResponseError with 2015: table 'test_order' is locked, please retry compaction later

[JackTan25]

standalone: full test pass 2000 times

[2023-11-27T08:51:11Z INFO  test_replace_recluster] ==========================
[2023-11-27T08:51:11Z INFO  test_replace_recluster] ====verify table state====
[2023-11-27T08:51:11Z INFO  test_replace_recluster] ==========================
[2023-11-27T08:51:11Z INFO  test_replace_recluster]                            
[2023-11-27T08:51:11Z INFO  test_replace_recluster]                            
[2023-11-27T08:51:11Z INFO  test_replace_recluster] number of successfully executed merge-into statements : 2000
[2023-11-27T08:51:11Z INFO  test_replace_recluster]                            
[2023-11-27T08:51:11Z INFO  test_replace_recluster]                            
[2023-11-27T08:51:11Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements
[2023-11-27T08:51:11Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements: client 2000000, server 2000000
[2023-11-27T08:51:11Z INFO  test_replace_recluster] CHECK: distinct ids: client 2000, server 2000
[2023-11-27T08:51:11Z INFO  test_replace_recluster] CHECK: value of correlated column
[2023-11-27T08:51:11Z INFO  test_replace_recluster] CHECK: full table scanning
[2023-11-27T08:51:11Z INFO  test_replace_recluster] ===========================
[2023-11-27T08:51:11Z INFO  test_replace_recluster] ======     PASSED      ====
[2023-11-27T08:51:11Z INFO  test_replace_recluster] ===========================
[2023-11-27T08:51:11Z INFO  test_replace_recluster]                            
[2023-11-27T08:51:11Z INFO  test_replace_recluster]                            
[2023-11-27T08:51:11Z INFO  test_replace_recluster] ========METRICS============
[2023-11-27T08:51:11Z INFO  test_replace_recluster] fuse_commit_mutation_unresolvable_conflict_total : 1606.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 23216546512.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 30906.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] fuse_replace_into_number_accumulated_merge_action_total : 0.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T08:51:11Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_count : 0.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_sum : 0.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":3792.0},{"less_than":50.0,"count":3792.0},{"less_than":100.0,"count":3792.0},{"less_than":250.0,"count":3792.0},{"less_than":500.0,"count":3792.0},{"less_than":1000.0,"count":3792.0},{"less_than":2500.0,"count":3792.0},{"less_than":5000.0,"count":3792.0},{"less_than":10000.0,"count":3792.0},{"less_than":20000.0,"count":3792.0},{"less_than":30000.0,"count":3792.0},{"less_than":60000.0,"count":3792.0},{"less_than":300000.0,"count":3792.0},{"less_than":600000.0,"count":3792.0},{"less_than":1800000.0,"count":3792.0},{"less_than":null,"count":3792.0}]
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 3792.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 3.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 5792.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 2855000.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2002.0},{"less_than":100.0,"count":2005.0},{"less_than":250.0,"count":2011.0},{"less_than":500.0,"count":2027.0},{"less_than":1000.0,"count":2056.0},{"less_than":2500.0,"count":2285.0},{"less_than":5000.0,"count":2285.0},{"less_than":10000.0,"count":2285.0},{"less_than":20000.0,"count":2285.0},{"less_than":30000.0,"count":2285.0},{"less_than":60000.0,"count":2285.0},{"less_than":300000.0,"count":2285.0},{"less_than":600000.0,"count":2285.0},{"less_than":1800000.0,"count":2285.0},{"less_than":null,"count":2285.0}]
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 2285.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 307340.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 48.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 48000.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":3792.0},{"less_than":50.0,"count":3792.0},{"less_than":100.0,"count":3792.0},{"less_than":250.0,"count":3792.0},{"less_than":500.0,"count":3792.0},{"less_than":1000.0,"count":3792.0},{"less_than":2500.0,"count":3792.0},{"less_than":5000.0,"count":3792.0},{"less_than":10000.0,"count":3792.0},{"less_than":20000.0,"count":3792.0},{"less_than":30000.0,"count":3792.0},{"less_than":60000.0,"count":3792.0},{"less_than":300000.0,"count":3792.0},{"less_than":600000.0,"count":3792.0},{"less_than":1800000.0,"count":3792.0},{"less_than":null,"count":3792.0}]
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 3792.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 985.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_matched_rows_total : 855000.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2000.0},{"less_than":100.0,"count":2000.0},{"less_than":250.0,"count":2000.0},{"less_than":500.0,"count":2000.0},{"less_than":1000.0,"count":2000.0},{"less_than":2500.0,"count":2000.0},{"less_than":5000.0,"count":2000.0},{"less_than":10000.0,"count":2000.0},{"less_than":20000.0,"count":2000.0},{"less_than":30000.0,"count":2000.0},{"less_than":60000.0,"count":2000.0},{"less_than":300000.0,"count":2000.0},{"less_than":600000.0,"count":2000.0},{"less_than":1800000.0,"count":2000.0},{"less_than":null,"count":2000.0}]
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_count : 2000.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_sum : 0.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 910.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 214728857.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":5792.0},{"less_than":50.0,"count":5792.0},{"less_than":100.0,"count":5792.0},{"less_than":250.0,"count":5792.0},{"less_than":500.0,"count":5792.0},{"less_than":1000.0,"count":5792.0},{"less_than":2500.0,"count":5792.0},{"less_than":5000.0,"count":5792.0},{"less_than":10000.0,"count":5792.0},{"less_than":20000.0,"count":5792.0},{"less_than":30000.0,"count":5792.0},{"less_than":60000.0,"count":5792.0},{"less_than":300000.0,"count":5792.0},{"less_than":600000.0,"count":5792.0},{"less_than":1800000.0,"count":5792.0},{"less_than":null,"count":5792.0}]
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 5792.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 2000000.0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] ===========================
[2023-11-27T08:51:11Z INFO  test_replace_recluster]                            
[2023-11-27T08:51:11Z INFO  test_replace_recluster]                            
[2023-11-27T08:51:11Z INFO  test_replace_recluster] ======CLUSTERING INFO======
[2023-11-27T08:51:11Z INFO  test_replace_recluster] cluster_key : (to_yyyymmdd(insert_time), id)
[2023-11-27T08:51:11Z INFO  test_replace_recluster] block_count: 11
[2023-11-27T08:51:11Z INFO  test_replace_recluster] constant_block_count: 1
[2023-11-27T08:51:11Z INFO  test_replace_recluster] unclustered_block_count: 0
[2023-11-27T08:51:11Z INFO  test_replace_recluster] average_overlaps: 7.0909
[2023-11-27T08:51:11Z INFO  test_replace_recluster] average_depth: 6.4545
[2023-11-27T08:51:11Z INFO  test_replace_recluster] block_depth_histogram: {"00001":1,"00007":10}
[2023-11-27T08:51:11Z INFO  test_replace_recluster] ===========================

[JackTan25]

distributed mode: full passed 2000time

[2023-11-27T09:55:20Z INFO  test_replace_recluster] ==========================
[2023-11-27T09:55:20Z INFO  test_replace_recluster] ====verify table state====
[2023-11-27T09:55:20Z INFO  test_replace_recluster] ==========================
[2023-11-27T09:55:20Z INFO  test_replace_recluster]                            
[2023-11-27T09:55:20Z INFO  test_replace_recluster]                            
[2023-11-27T09:55:20Z INFO  test_replace_recluster] number of successfully executed merge-into statements : 2000
[2023-11-27T09:55:20Z INFO  test_replace_recluster]                            
[2023-11-27T09:55:20Z INFO  test_replace_recluster]                            
[2023-11-27T09:55:20Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements
[2023-11-27T09:55:20Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements: client 2000000, server 2000000
[2023-11-27T09:55:20Z INFO  test_replace_recluster] CHECK: distinct ids: client 2000, server 2000
[2023-11-27T09:55:20Z INFO  test_replace_recluster] CHECK: value of correlated column
[2023-11-27T09:55:20Z INFO  test_replace_recluster] CHECK: full table scanning
[2023-11-27T09:55:20Z INFO  test_replace_recluster] ===========================
[2023-11-27T09:55:20Z INFO  test_replace_recluster] ======     PASSED      ====
[2023-11-27T09:55:20Z INFO  test_replace_recluster] ===========================
[2023-11-27T09:55:20Z INFO  test_replace_recluster]                            
[2023-11-27T09:55:20Z INFO  test_replace_recluster]                            
[2023-11-27T09:55:20Z INFO  test_replace_recluster] ========METRICS============
[2023-11-27T09:55:20Z INFO  test_replace_recluster] fuse_commit_mutation_unresolvable_conflict_total : 1530.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 12404898371.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 10521118234.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 12263.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 17676.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] fuse_replace_into_number_accumulated_merge_action_total : 0.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_count : 0.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_sum : 0.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":2287.0},{"less_than":50.0,"count":2287.0},{"less_than":100.0,"count":2287.0},{"less_than":250.0,"count":2287.0},{"less_than":500.0,"count":2287.0},{"less_than":1000.0,"count":2287.0},{"less_than":2500.0,"count":2287.0},{"less_than":5000.0,"count":2287.0},{"less_than":10000.0,"count":2287.0},{"less_than":20000.0,"count":2287.0},{"less_than":30000.0,"count":2287.0},{"less_than":60000.0,"count":2287.0},{"less_than":300000.0,"count":2287.0},{"less_than":600000.0,"count":2287.0},{"less_than":1800000.0,"count":2287.0},{"less_than":null,"count":2287.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":1533.0},{"less_than":50.0,"count":1533.0},{"less_than":100.0,"count":1533.0},{"less_than":250.0,"count":1533.0},{"less_than":500.0,"count":1533.0},{"less_than":1000.0,"count":1533.0},{"less_than":2500.0,"count":1533.0},{"less_than":5000.0,"count":1533.0},{"less_than":10000.0,"count":1533.0},{"less_than":20000.0,"count":1533.0},{"less_than":30000.0,"count":1533.0},{"less_than":60000.0,"count":1533.0},{"less_than":300000.0,"count":1533.0},{"less_than":600000.0,"count":1533.0},{"less_than":1800000.0,"count":1533.0},{"less_than":null,"count":1533.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 2287.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 1533.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 1.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 0.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 2287.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 3533.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 524124.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 2330876.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":2031.0},{"less_than":50.0,"count":2079.0},{"less_than":100.0,"count":2093.0},{"less_than":250.0,"count":2099.0},{"less_than":500.0,"count":2099.0},{"less_than":1000.0,"count":2105.0},{"less_than":2500.0,"count":2285.0},{"less_than":5000.0,"count":2285.0},{"less_than":10000.0,"count":2285.0},{"less_than":20000.0,"count":2285.0},{"less_than":30000.0,"count":2285.0},{"less_than":60000.0,"count":2285.0},{"less_than":300000.0,"count":2285.0},{"less_than":600000.0,"count":2285.0},{"less_than":1800000.0,"count":2285.0},{"less_than":null,"count":2285.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":2026.0},{"less_than":50.0,"count":2027.0},{"less_than":100.0,"count":2030.0},{"less_than":250.0,"count":2037.0},{"less_than":500.0,"count":2049.0},{"less_than":1000.0,"count":2083.0},{"less_than":2500.0,"count":2285.0},{"less_than":5000.0,"count":2285.0},{"less_than":10000.0,"count":2285.0},{"less_than":20000.0,"count":2285.0},{"less_than":30000.0,"count":2285.0},{"less_than":60000.0,"count":2285.0},{"less_than":300000.0,"count":2285.0},{"less_than":600000.0,"count":2285.0},{"less_than":1800000.0,"count":2285.0},{"less_than":null,"count":2285.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 2285.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 2285.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 255478.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 310483.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 45.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 0.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 45000.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 0.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_apply_row_number_total : 2000000.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_deduplicate_row_number_total : 4855000.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_empty_row_number_total : 27.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 2524124.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 330876.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_hashtable_empty_block_total : 285.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_hashtable_fetch_row_number_total : 2000000.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_rows_total : 2000000.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_total : 2000.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_init_unique_number_total : 2503204.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_distributed_new_set_len_total : 2000000.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":2287.0},{"less_than":50.0,"count":2287.0},{"less_than":100.0,"count":2287.0},{"less_than":250.0,"count":2287.0},{"less_than":500.0,"count":2287.0},{"less_than":1000.0,"count":2287.0},{"less_than":2500.0,"count":2287.0},{"less_than":5000.0,"count":2287.0},{"less_than":10000.0,"count":2287.0},{"less_than":20000.0,"count":2287.0},{"less_than":30000.0,"count":2287.0},{"less_than":60000.0,"count":2287.0},{"less_than":300000.0,"count":2287.0},{"less_than":600000.0,"count":2287.0},{"less_than":1800000.0,"count":2287.0},{"less_than":null,"count":2287.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":1533.0},{"less_than":50.0,"count":1533.0},{"less_than":100.0,"count":1533.0},{"less_than":250.0,"count":1533.0},{"less_than":500.0,"count":1533.0},{"less_than":1000.0,"count":1533.0},{"less_than":2500.0,"count":1533.0},{"less_than":5000.0,"count":1533.0},{"less_than":10000.0,"count":1533.0},{"less_than":20000.0,"count":1533.0},{"less_than":30000.0,"count":1533.0},{"less_than":60000.0,"count":1533.0},{"less_than":300000.0,"count":1533.0},{"less_than":600000.0,"count":1533.0},{"less_than":1800000.0,"count":1533.0},{"less_than":null,"count":1533.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 2287.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 1533.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 690.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 276.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_matched_rows_total : 524124.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_matched_rows_total : 330876.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2000.0},{"less_than":100.0,"count":2000.0},{"less_than":250.0,"count":2000.0},{"less_than":500.0,"count":2000.0},{"less_than":1000.0,"count":2000.0},{"less_than":2500.0,"count":2000.0},{"less_than":5000.0,"count":2000.0},{"less_than":10000.0,"count":2000.0},{"less_than":20000.0,"count":2000.0},{"less_than":30000.0,"count":2000.0},{"less_than":60000.0,"count":2000.0},{"less_than":300000.0,"count":2000.0},{"less_than":600000.0,"count":2000.0},{"less_than":1800000.0,"count":2000.0},{"less_than":null,"count":2000.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_count : 2000.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_sum : 0.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 563.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 295.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 122579233.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 90308767.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":4546.0},{"less_than":50.0,"count":4546.0},{"less_than":100.0,"count":4546.0},{"less_than":250.0,"count":4546.0},{"less_than":500.0,"count":4546.0},{"less_than":1000.0,"count":4546.0},{"less_than":2500.0,"count":4546.0},{"less_than":5000.0,"count":4546.0},{"less_than":10000.0,"count":4546.0},{"less_than":20000.0,"count":4546.0},{"less_than":30000.0,"count":4546.0},{"less_than":60000.0,"count":4546.0},{"less_than":300000.0,"count":4546.0},{"less_than":600000.0,"count":4546.0},{"less_than":1800000.0,"count":4546.0},{"less_than":null,"count":4546.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":3817.0},{"less_than":50.0,"count":3817.0},{"less_than":100.0,"count":3817.0},{"less_than":250.0,"count":3817.0},{"less_than":500.0,"count":3817.0},{"less_than":1000.0,"count":3817.0},{"less_than":2500.0,"count":3817.0},{"less_than":5000.0,"count":3817.0},{"less_than":10000.0,"count":3817.0},{"less_than":20000.0,"count":3817.0},{"less_than":30000.0,"count":3817.0},{"less_than":60000.0,"count":3817.0},{"less_than":300000.0,"count":3817.0},{"less_than":600000.0,"count":3817.0},{"less_than":1800000.0,"count":3817.0},{"less_than":null,"count":3817.0}]
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 4546.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 3817.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 2524124.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 2330876.0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] ===========================
[2023-11-27T09:55:20Z INFO  test_replace_recluster]                            
[2023-11-27T09:55:20Z INFO  test_replace_recluster]                            
[2023-11-27T09:55:20Z INFO  test_replace_recluster] ======CLUSTERING INFO======
[2023-11-27T09:55:20Z INFO  test_replace_recluster] cluster_key : (to_yyyymmdd(insert_time), id)
[2023-11-27T09:55:20Z INFO  test_replace_recluster] block_count: 9
[2023-11-27T09:55:20Z INFO  test_replace_recluster] constant_block_count: 0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] unclustered_block_count: 0
[2023-11-27T09:55:20Z INFO  test_replace_recluster] average_overlaps: 6.6667
[2023-11-27T09:55:20Z INFO  test_replace_recluster] average_depth: 6
[2023-11-27T09:55:20Z INFO  test_replace_recluster] block_depth_histogram: {"00006":9}
[2023-11-27T09:55:20Z INFO  test_replace_recluster] ===========================

[JackTan25]

standalone: test_delete passed 2000 times

[2023-11-27T10:24:07Z INFO  test_replace_recluster] ==========================
[2023-11-27T10:24:07Z INFO  test_replace_recluster] ====verify table state====
[2023-11-27T10:24:07Z INFO  test_replace_recluster] ==========================
[2023-11-27T10:24:07Z INFO  test_replace_recluster]                            
[2023-11-27T10:24:07Z INFO  test_replace_recluster]                            
[2023-11-27T10:24:07Z INFO  test_replace_recluster] number of successfully executed merge-into statements : 2000
[2023-11-27T10:24:07Z INFO  test_replace_recluster]                            
[2023-11-27T10:24:07Z INFO  test_replace_recluster]                            
[2023-11-27T10:24:07Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements
[2023-11-27T10:24:07Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements: client 2000000, server 0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] CHECK: full table scanning
[2023-11-27T10:24:07Z INFO  test_replace_recluster] ===========================
[2023-11-27T10:24:07Z INFO  test_replace_recluster] ======     PASSED      ====
[2023-11-27T10:24:07Z INFO  test_replace_recluster] ===========================
[2023-11-27T10:24:07Z INFO  test_replace_recluster]                            
[2023-11-27T10:24:07Z INFO  test_replace_recluster]                            
[2023-11-27T10:24:07Z INFO  test_replace_recluster] ========METRICS============
[2023-11-27T10:24:07Z INFO  test_replace_recluster] fuse_commit_mutation_unresolvable_conflict_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 200114237.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 2000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] fuse_replace_into_number_accumulated_merge_action_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_count : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_sum : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2000.0},{"less_than":100.0,"count":2000.0},{"less_than":250.0,"count":2000.0},{"less_than":500.0,"count":2000.0},{"less_than":1000.0,"count":2000.0},{"less_than":2500.0,"count":2000.0},{"less_than":5000.0,"count":2000.0},{"less_than":10000.0,"count":2000.0},{"less_than":20000.0,"count":2000.0},{"less_than":30000.0,"count":2000.0},{"less_than":60000.0,"count":2000.0},{"less_than":300000.0,"count":2000.0},{"less_than":600000.0,"count":2000.0},{"less_than":1800000.0,"count":2000.0},{"less_than":null,"count":2000.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 2000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 1.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 2000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 2000000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 4000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 2757.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 2000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 2000000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_apply_row_number_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_deduplicate_row_number_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_empty_row_number_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_hashtable_empty_block_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_hashtable_fetch_row_number_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_rows_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_init_unique_number_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_distributed_new_set_len_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_matched_rows_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_matched_rows_total : 2000000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2000.0},{"less_than":100.0,"count":2000.0},{"less_than":250.0,"count":2000.0},{"less_than":500.0,"count":2000.0},{"less_than":1000.0,"count":2000.0},{"less_than":2500.0,"count":2000.0},{"less_than":5000.0,"count":2000.0},{"less_than":10000.0,"count":2000.0},{"less_than":20000.0,"count":2000.0},{"less_than":30000.0,"count":2000.0},{"less_than":60000.0,"count":2000.0},{"less_than":300000.0,"count":2000.0},{"less_than":600000.0,"count":2000.0},{"less_than":1800000.0,"count":2000.0},{"less_than":null,"count":2000.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_count : 2000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_sum : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 4000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 2000000.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 0.0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] ===========================
[2023-11-27T10:24:07Z INFO  test_replace_recluster]                            
[2023-11-27T10:24:07Z INFO  test_replace_recluster]                            
[2023-11-27T10:24:07Z INFO  test_replace_recluster] ======CLUSTERING INFO======
[2023-11-27T10:24:07Z INFO  test_replace_recluster] cluster_key : (to_yyyymmdd(insert_time), id)
[2023-11-27T10:24:07Z INFO  test_replace_recluster] block_count: 0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] constant_block_count: 0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] unclustered_block_count: 0
[2023-11-27T10:24:07Z INFO  test_replace_recluster] average_overlaps: NaN
[2023-11-27T10:24:07Z INFO  test_replace_recluster] average_depth: NaN
[2023-11-27T10:24:07Z INFO  test_replace_recluster] block_depth_histogram: {}
[2023-11-27T10:24:07Z INFO  test_replace_recluster] ===========================

[JackTan25]

distributed mode: test_delete passed 2000 times

[2023-11-27T11:03:16Z INFO  test_replace_recluster] ==========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ====verify table state====
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ==========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster] number of successfully executed merge-into statements : 2000
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements
[2023-11-27T11:03:16Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements: client 2000000, server 0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] CHECK: full table scanning
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ===========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ======     PASSED      ====
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ===========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ========METRICS============
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_commit_mutation_unresolvable_conflict_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 200127763.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_replace_into_number_accumulated_merge_action_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_count : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2000.0},{"less_than":100.0,"count":2000.0},{"less_than":250.0,"count":2000.0},{"less_than":500.0,"count":2000.0},{"less_than":1000.0,"count":2000.0},{"less_than":2500.0,"count":2000.0},{"less_than":5000.0,"count":2000.0},{"less_than":10000.0,"count":2000.0},{"less_than":20000.0,"count":2000.0},{"less_than":30000.0,"count":2000.0},{"less_than":60000.0,"count":2000.0},{"less_than":300000.0,"count":2000.0},{"less_than":600000.0,"count":2000.0},{"less_than":1800000.0,"count":2000.0},{"less_than":null,"count":2000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 10.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 4000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 4000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 2870.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_apply_row_number_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_deduplicate_row_number_total : 6000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_empty_row_number_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 4000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_hashtable_empty_block_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_hashtable_fetch_row_number_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_rows_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_init_unique_number_total : 3997000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_new_set_len_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_rows_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_rows_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2000.0},{"less_than":100.0,"count":2000.0},{"less_than":250.0,"count":2000.0},{"less_than":500.0,"count":2000.0},{"less_than":1000.0,"count":2000.0},{"less_than":2500.0,"count":2000.0},{"less_than":5000.0,"count":2000.0},{"less_than":10000.0,"count":2000.0},{"less_than":20000.0,"count":2000.0},{"less_than":30000.0,"count":2000.0},{"less_than":60000.0,"count":2000.0},{"less_than":300000.0,"count":2000.0},{"less_than":600000.0,"count":2000.0},{"less_than":1800000.0,"count":2000.0},{"less_than":null,"count":2000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_count : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 4000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 4000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 4000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ===========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ======CLUSTERING INFO======
[2023-11-27T11:03:16Z INFO  test_replace_recluster] cluster_key : (to_yyyymmdd(insert_time), id)
[2023-11-27T11:03:16Z INFO  test_replace_recluster] block_count: 0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] constant_block_count: 0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] unclustered_block_count: 0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] average_overlaps: NaN
[2023-11-27T11:03:16Z INFO  test_replace_recluster] average_depth: NaN
[2023-11-27T11:03:16Z INFO  test_replace_recluster] block_depth_histogram: {}
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ===========================

[JackTan25]

standalone mode: insert only test passed 2000 times

[2023-11-27T11:38:28Z INFO  test_replace_recluster] ==========================
[2023-11-27T11:38:28Z INFO  test_replace_recluster] ====verify table state====
[2023-11-27T11:38:28Z INFO  test_replace_recluster] ==========================
[2023-11-27T11:38:28Z INFO  test_replace_recluster]                            
[2023-11-27T11:38:28Z INFO  test_replace_recluster]                            
[2023-11-27T11:38:28Z INFO  test_replace_recluster] number of successfully executed merge-into statements : 2000
[2023-11-27T11:38:28Z INFO  test_replace_recluster]                            
[2023-11-27T11:38:28Z INFO  test_replace_recluster]                            
[2023-11-27T11:38:28Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements
[2023-11-27T11:38:28Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements: client 2000000, server 2000000
[2023-11-27T11:38:28Z INFO  test_replace_recluster] CHECK: distinct ids: client 2000, server 2000
[2023-11-27T11:38:28Z INFO  test_replace_recluster] CHECK: value of correlated column
[2023-11-27T11:38:28Z INFO  test_replace_recluster] CHECK: full table scanning
[2023-11-27T11:38:29Z INFO  test_replace_recluster] ===========================
[2023-11-27T11:38:29Z INFO  test_replace_recluster] ======     PASSED      ====
[2023-11-27T11:38:29Z INFO  test_replace_recluster] ===========================
[2023-11-27T11:38:29Z INFO  test_replace_recluster]                            
[2023-11-27T11:38:29Z INFO  test_replace_recluster]                            
[2023-11-27T11:38:29Z INFO  test_replace_recluster] ========METRICS============
[2023-11-27T11:38:29Z INFO  test_replace_recluster] fuse_commit_mutation_unresolvable_conflict_total : 1589.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 2132770498.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 6669.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] fuse_replace_into_number_accumulated_merge_action_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_count : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 2000.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 2000000.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_apply_row_number_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_deduplicate_row_number_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_empty_row_number_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_hashtable_empty_block_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_hashtable_fetch_row_number_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_rows_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_init_unique_number_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_distributed_new_set_len_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_matched_rows_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_matched_rows_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2000.0},{"less_than":100.0,"count":2000.0},{"less_than":250.0,"count":2000.0},{"less_than":500.0,"count":2000.0},{"less_than":1000.0,"count":2000.0},{"less_than":2500.0,"count":2000.0},{"less_than":5000.0,"count":2000.0},{"less_than":10000.0,"count":2000.0},{"less_than":20000.0,"count":2000.0},{"less_than":30000.0,"count":2000.0},{"less_than":60000.0,"count":2000.0},{"less_than":300000.0,"count":2000.0},{"less_than":600000.0,"count":2000.0},{"less_than":1800000.0,"count":2000.0},{"less_than":null,"count":2000.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_count : 2000.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 0.0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] ===========================
[2023-11-27T11:38:29Z INFO  test_replace_recluster]                            
[2023-11-27T11:38:29Z INFO  test_replace_recluster]                            
[2023-11-27T11:38:29Z INFO  test_replace_recluster] ======CLUSTERING INFO======
[2023-11-27T11:38:29Z INFO  test_replace_recluster] cluster_key : (to_yyyymmdd(insert_time), id)
[2023-11-27T11:38:29Z INFO  test_replace_recluster] block_count: 9
[2023-11-27T11:38:29Z INFO  test_replace_recluster] constant_block_count: 0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] unclustered_block_count: 0
[2023-11-27T11:38:29Z INFO  test_replace_recluster] average_overlaps: 7.3333
[2023-11-27T11:38:29Z INFO  test_replace_recluster] average_depth: 7
[2023-11-27T11:38:29Z INFO  test_replace_recluster] block_depth_histogram: {"00007":9}
[2023-11-27T11:38:29Z INFO  test_replace_recluster] ===========================

[JackTan25]

distributed mode: insert-only test passed 2000 times

[2023-11-27T11:03:16Z INFO  test_replace_recluster] ==========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ====verify table state====
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ==========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster] number of successfully executed merge-into statements : 2000
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements
[2023-11-27T11:03:16Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements: client 2000000, server 0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] CHECK: full table scanning
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ===========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ======     PASSED      ====
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ===========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ========METRICS============
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_commit_mutation_unresolvable_conflict_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 200127763.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_replace_into_number_accumulated_merge_action_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_count : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] fuse_replace_into_time_accumulated_merge_action_ms_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2000.0},{"less_than":100.0,"count":2000.0},{"less_than":250.0,"count":2000.0},{"less_than":500.0,"count":2000.0},{"less_than":1000.0,"count":2000.0},{"less_than":2500.0,"count":2000.0},{"less_than":5000.0,"count":2000.0},{"less_than":10000.0,"count":2000.0},{"less_than":20000.0,"count":2000.0},{"less_than":30000.0,"count":2000.0},{"less_than":60000.0,"count":2000.0},{"less_than":300000.0,"count":2000.0},{"less_than":600000.0,"count":2000.0},{"less_than":1800000.0,"count":2000.0},{"less_than":null,"count":2000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 10.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 4000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 4000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 2870.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_apply_row_number_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_deduplicate_row_number_total : 6000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_empty_row_number_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 4000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_hashtable_empty_block_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_hashtable_fetch_row_number_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_rows_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_hashtable_push_null_block_total : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_init_unique_number_total : 3997000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_distributed_new_set_len_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":0.0},{"less_than":50.0,"count":0.0},{"less_than":100.0,"count":0.0},{"less_than":250.0,"count":0.0},{"less_than":500.0,"count":0.0},{"less_than":1000.0,"count":0.0},{"less_than":2500.0,"count":0.0},{"less_than":5000.0,"count":0.0},{"less_than":10000.0,"count":0.0},{"less_than":20000.0,"count":0.0},{"less_than":30000.0,"count":0.0},{"less_than":60000.0,"count":0.0},{"less_than":300000.0,"count":0.0},{"less_than":600000.0,"count":0.0},{"less_than":1800000.0,"count":0.0},{"less_than":null,"count":0.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_rows_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_matched_rows_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds : [{"less_than":10.0,"count":2000.0},{"less_than":50.0,"count":2000.0},{"less_than":100.0,"count":2000.0},{"less_than":250.0,"count":2000.0},{"less_than":500.0,"count":2000.0},{"less_than":1000.0,"count":2000.0},{"less_than":2500.0,"count":2000.0},{"less_than":5000.0,"count":2000.0},{"less_than":10000.0,"count":2000.0},{"less_than":20000.0,"count":2000.0},{"less_than":30000.0,"count":2000.0},{"less_than":60000.0,"count":2000.0},{"less_than":300000.0,"count":2000.0},{"less_than":600000.0,"count":2000.0},{"less_than":1800000.0,"count":2000.0},{"less_than":null,"count":2000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_count : 2000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_not_matched_operation_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds : [{"less_than":10.0,"count":4000.0},{"less_than":50.0,"count":4000.0},{"less_than":100.0,"count":4000.0},{"less_than":250.0,"count":4000.0},{"less_than":500.0,"count":4000.0},{"less_than":1000.0,"count":4000.0},{"less_than":2500.0,"count":4000.0},{"less_than":5000.0,"count":4000.0},{"less_than":10000.0,"count":4000.0},{"less_than":20000.0,"count":4000.0},{"less_than":30000.0,"count":4000.0},{"less_than":60000.0,"count":4000.0},{"less_than":300000.0,"count":4000.0},{"less_than":600000.0,"count":4000.0},{"less_than":1800000.0,"count":4000.0},{"less_than":null,"count":4000.0}]
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 4000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds_count : 4000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_split_milliseconds_sum : 0.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 4000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] merge_into_unmatched_rows_total : 2000000.0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ===========================
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster]                            
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ======CLUSTERING INFO======
[2023-11-27T11:03:16Z INFO  test_replace_recluster] cluster_key : (to_yyyymmdd(insert_time), id)
[2023-11-27T11:03:16Z INFO  test_replace_recluster] block_count: 0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] constant_block_count: 0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] unclustered_block_count: 0
[2023-11-27T11:03:16Z INFO  test_replace_recluster] average_overlaps: NaN
[2023-11-27T11:03:16Z INFO  test_replace_recluster] average_depth: NaN
[2023-11-27T11:03:16Z INFO  test_replace_recluster] block_depth_histogram: {}
[2023-11-27T11:03:16Z INFO  test_replace_recluster] ===========================

[JackTan25]

stanalone mode: matched only test, iteration 1000 and succeed 15 times because of the conflicts (see the test_script :https://github.com/JackTan25/test-scripts matched only branch) and passed tests.

[2023-11-27T15:36:13Z INFO  test_replace_recluster] ==========================
[2023-11-27T15:36:13Z INFO  test_replace_recluster] ====verify table state====
[2023-11-27T15:36:13Z INFO  test_replace_recluster] ==========================
[2023-11-27T15:36:13Z INFO  test_replace_recluster]                            
[2023-11-27T15:36:13Z INFO  test_replace_recluster]                            
[2023-11-27T15:36:13Z INFO  test_replace_recluster] number of successfully executed merge-into statements : 15
[2023-11-27T15:36:13Z INFO  test_replace_recluster]                            
[2023-11-27T15:36:13Z INFO  test_replace_recluster]                            
[2023-11-27T15:36:13Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements
[2023-11-27T15:36:13Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements: client 15000, server 1000000
[2023-11-27T15:36:13Z INFO  test_replace_recluster] CHECK: distinct ids: client 15, server 15
[2023-11-27T15:36:13Z INFO  test_replace_recluster] CHECK: value of correlated column
[2023-11-27T15:36:13Z INFO  test_replace_recluster] CHECK: full table scanning
[2023-11-27T15:36:13Z INFO  test_replace_recluster] ===========================
[2023-11-27T15:36:13Z INFO  test_replace_recluster] ======     PASSED      ====
[2023-11-27T15:36:13Z INFO  test_replace_recluster] ===========================
[2023-11-27T15:36:13Z INFO  test_replace_recluster]                            
[2023-11-27T15:36:13Z INFO  test_replace_recluster]                            
[2023-11-27T15:36:13Z INFO  test_replace_recluster] ========METRICS============
[2023-11-27T15:36:13Z INFO  test_replace_recluster] fuse_commit_mutation_unresolvable_conflict_total : 1128.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 4934128252.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 6810.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":1935.0},{"less_than":50.0,"count":1935.0},{"less_than":100.0,"count":1935.0},{"less_than":250.0,"count":1935.0},{"less_than":500.0,"count":1935.0},{"less_than":1000.0,"count":1935.0},{"less_than":2500.0,"count":1935.0},{"less_than":5000.0,"count":1935.0},{"less_than":10000.0,"count":1935.0},{"less_than":20000.0,"count":1935.0},{"less_than":30000.0,"count":1935.0},{"less_than":60000.0,"count":1935.0},{"less_than":300000.0,"count":1935.0},{"less_than":600000.0,"count":1935.0},{"less_than":1800000.0,"count":1935.0},{"less_than":null,"count":1935.0}]
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 1935.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 0.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 1935.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 1424000.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":999.0},{"less_than":50.0,"count":1003.0},{"less_than":100.0,"count":1005.0},{"less_than":250.0,"count":1015.0},{"less_than":500.0,"count":1034.0},{"less_than":1000.0,"count":1075.0},{"less_than":2500.0,"count":1142.0},{"less_than":5000.0,"count":1142.0},{"less_than":10000.0,"count":1142.0},{"less_than":20000.0,"count":1142.0},{"less_than":30000.0,"count":1142.0},{"less_than":60000.0,"count":1142.0},{"less_than":300000.0,"count":1142.0},{"less_than":600000.0,"count":1142.0},{"less_than":1800000.0,"count":1142.0},{"less_than":null,"count":1142.0}]
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 1142.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 156045.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 1068.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 1068000.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":1935.0},{"less_than":50.0,"count":1935.0},{"less_than":100.0,"count":1935.0},{"less_than":250.0,"count":1935.0},{"less_than":500.0,"count":1935.0},{"less_than":1000.0,"count":1935.0},{"less_than":2500.0,"count":1935.0},{"less_than":5000.0,"count":1935.0},{"less_than":10000.0,"count":1935.0},{"less_than":20000.0,"count":1935.0},{"less_than":30000.0,"count":1935.0},{"less_than":60000.0,"count":1935.0},{"less_than":300000.0,"count":1935.0},{"less_than":600000.0,"count":1935.0},{"less_than":1800000.0,"count":1935.0},{"less_than":null,"count":1935.0}]
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 1935.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 236.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 47162000.0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] ===========================
[2023-11-27T15:36:13Z INFO  test_replace_recluster]                            
[2023-11-27T15:36:13Z INFO  test_replace_recluster]                            
[2023-11-27T15:36:13Z INFO  test_replace_recluster] ======CLUSTERING INFO======
[2023-11-27T15:36:13Z INFO  test_replace_recluster] cluster_key : (to_yyyymmdd(insert_time), id)
[2023-11-27T15:36:13Z INFO  test_replace_recluster] block_count: 9
[2023-11-27T15:36:13Z INFO  test_replace_recluster] constant_block_count: 0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] unclustered_block_count: 0
[2023-11-27T15:36:13Z INFO  test_replace_recluster] average_overlaps: 8
[2023-11-27T15:36:13Z INFO  test_replace_recluster] average_depth: 9
[2023-11-27T15:36:13Z INFO  test_replace_recluster] block_depth_histogram: {"00009":9}
[2023-11-27T15:36:13Z INFO  test_replace_recluster] ===========================

[JackTan25]

distributed mode: matched only test, iteration 1000 and succeed 12 times because of the conflict (see the test_script :https://github.com/JackTan25/test-scripts matched only branch) and passed tests.

2023-11-27T15:59:49Z INFO  test_replace_recluster] ==========================
[2023-11-27T15:59:49Z INFO  test_replace_recluster] ====verify table state====
[2023-11-27T15:59:49Z INFO  test_replace_recluster] ==========================
[2023-11-27T15:59:49Z INFO  test_replace_recluster]                            
[2023-11-27T15:59:49Z INFO  test_replace_recluster]                            
[2023-11-27T15:59:49Z INFO  test_replace_recluster] number of successfully executed merge-into statements : 12
[2023-11-27T15:59:49Z INFO  test_replace_recluster]                            
[2023-11-27T15:59:49Z INFO  test_replace_recluster]                            
[2023-11-27T15:59:49Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements
[2023-11-27T15:59:49Z INFO  test_replace_recluster] CHECK: value of successfully executed merge-into statements: client 12000, server 1000000
[2023-11-27T15:59:49Z INFO  test_replace_recluster] CHECK: distinct ids: client 12, server 12
[2023-11-27T15:59:50Z INFO  test_replace_recluster] CHECK: value of correlated column
[2023-11-27T15:59:50Z INFO  test_replace_recluster] CHECK: full table scanning
[2023-11-27T15:59:50Z INFO  test_replace_recluster] ===========================
[2023-11-27T15:59:50Z INFO  test_replace_recluster] ======     PASSED      ====
[2023-11-27T15:59:50Z INFO  test_replace_recluster] ===========================
[2023-11-27T15:59:50Z INFO  test_replace_recluster]                            
[2023-11-27T15:59:50Z INFO  test_replace_recluster]                            
[2023-11-27T15:59:50Z INFO  test_replace_recluster] ========METRICS============
[2023-11-27T15:59:50Z INFO  test_replace_recluster] fuse_commit_mutation_unresolvable_conflict_total : 1148.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 178977270.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] fuse_remote_io_read_bytes_after_merged_total : 4786416182.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 344.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] fuse_remote_io_seeks_after_merged_total : 6535.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":176.0},{"less_than":50.0,"count":176.0},{"less_than":100.0,"count":176.0},{"less_than":250.0,"count":176.0},{"less_than":500.0,"count":176.0},{"less_than":1000.0,"count":176.0},{"less_than":2500.0,"count":176.0},{"less_than":5000.0,"count":176.0},{"less_than":10000.0,"count":176.0},{"less_than":20000.0,"count":176.0},{"less_than":30000.0,"count":176.0},{"less_than":60000.0,"count":176.0},{"less_than":300000.0,"count":176.0},{"less_than":600000.0,"count":176.0},{"less_than":1800000.0,"count":176.0},{"less_than":null,"count":176.0}]
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds : [{"less_than":10.0,"count":1760.0},{"less_than":50.0,"count":1760.0},{"less_than":100.0,"count":1760.0},{"less_than":250.0,"count":1760.0},{"less_than":500.0,"count":1760.0},{"less_than":1000.0,"count":1760.0},{"less_than":2500.0,"count":1760.0},{"less_than":5000.0,"count":1760.0},{"less_than":10000.0,"count":1760.0},{"less_than":20000.0,"count":1760.0},{"less_than":30000.0,"count":1760.0},{"less_than":60000.0,"count":1760.0},{"less_than":300000.0,"count":1760.0},{"less_than":600000.0,"count":1760.0},{"less_than":1800000.0,"count":1760.0},{"less_than":null,"count":1760.0}]
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 176.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_count : 1760.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 1.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_accumulate_milliseconds_sum : 2.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 176.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_append_blocks_counter_total : 1760.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 170000.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_append_blocks_rows_counter_total : 1252000.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":1081.0},{"less_than":50.0,"count":1124.0},{"less_than":100.0,"count":1131.0},{"less_than":250.0,"count":1137.0},{"less_than":500.0,"count":1137.0},{"less_than":1000.0,"count":1141.0},{"less_than":2500.0,"count":1142.0},{"less_than":5000.0,"count":1142.0},{"less_than":10000.0,"count":1142.0},{"less_than":20000.0,"count":1142.0},{"less_than":30000.0,"count":1142.0},{"less_than":60000.0,"count":1142.0},{"less_than":300000.0,"count":1142.0},{"less_than":600000.0,"count":1142.0},{"less_than":1800000.0,"count":1142.0},{"less_than":null,"count":1142.0}]
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_apply_milliseconds : [{"less_than":10.0,"count":1006.0},{"less_than":50.0,"count":1009.0},{"less_than":100.0,"count":1013.0},{"less_than":250.0,"count":1019.0},{"less_than":500.0,"count":1038.0},{"less_than":1000.0,"count":1071.0},{"less_than":2500.0,"count":1142.0},{"less_than":5000.0,"count":1142.0},{"less_than":10000.0,"count":1142.0},{"less_than":20000.0,"count":1142.0},{"less_than":30000.0,"count":1142.0},{"less_than":60000.0,"count":1142.0},{"less_than":300000.0,"count":1142.0},{"less_than":600000.0,"count":1142.0},{"less_than":1800000.0,"count":1142.0},{"less_than":null,"count":1142.0}]
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 1142.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_apply_milliseconds_count : 1142.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 6894.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_apply_milliseconds_sum : 154941.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 81.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_deleted_blocks_counter_total : 986.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 81000.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_deleted_blocks_rows_counter_total : 986000.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 1163000.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_distributed_generate_row_numbers_total : 259000.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":176.0},{"less_than":50.0,"count":176.0},{"less_than":100.0,"count":176.0},{"less_than":250.0,"count":176.0},{"less_than":500.0,"count":176.0},{"less_than":1000.0,"count":176.0},{"less_than":2500.0,"count":176.0},{"less_than":5000.0,"count":176.0},{"less_than":10000.0,"count":176.0},{"less_than":20000.0,"count":176.0},{"less_than":30000.0,"count":176.0},{"less_than":60000.0,"count":176.0},{"less_than":300000.0,"count":176.0},{"less_than":600000.0,"count":176.0},{"less_than":1800000.0,"count":176.0},{"less_than":null,"count":176.0}]
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds : [{"less_than":10.0,"count":1760.0},{"less_than":50.0,"count":1760.0},{"less_than":100.0,"count":1760.0},{"less_than":250.0,"count":1760.0},{"less_than":500.0,"count":1760.0},{"less_than":1000.0,"count":1760.0},{"less_than":2500.0,"count":1760.0},{"less_than":5000.0,"count":1760.0},{"less_than":10000.0,"count":1760.0},{"less_than":20000.0,"count":1760.0},{"less_than":30000.0,"count":1760.0},{"less_than":60000.0,"count":1760.0},{"less_than":300000.0,"count":1760.0},{"less_than":600000.0,"count":1760.0},{"less_than":1800000.0,"count":1760.0},{"less_than":null,"count":1760.0}]
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 176.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_count : 1760.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_matched_operation_milliseconds_sum : 0.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 79.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_replace_blocks_counter_total : 157.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 45731000.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] merge_into_replace_blocks_rows_counter_total : 1709000.0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] ===========================
[2023-11-27T15:59:50Z INFO  test_replace_recluster]                            
[2023-11-27T15:59:50Z INFO  test_replace_recluster]                            
[2023-11-27T15:59:50Z INFO  test_replace_recluster] ======CLUSTERING INFO======
[2023-11-27T15:59:50Z INFO  test_replace_recluster] cluster_key : (to_yyyymmdd(insert_time), id)
[2023-11-27T15:59:50Z INFO  test_replace_recluster] block_count: 8
[2023-11-27T15:59:50Z INFO  test_replace_recluster] constant_block_count: 0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] unclustered_block_count: 0
[2023-11-27T15:59:50Z INFO  test_replace_recluster] average_overlaps: 7
[2023-11-27T15:59:50Z INFO  test_replace_recluster] average_depth: 8
[2023-11-27T15:59:50Z INFO  test_replace_recluster] block_depth_histogram: {"00008":8}
[2023-11-27T15:59:50Z INFO  test_replace_recluster] ===========================

[JackTan25]

logs:logs.zip

[JackTan25]

@dantengsky Tests have passed and we can get expected performance. please review this long delay pr, thanks.

[github-actions[bot]]

Docker Image for PR

• tag: pr-13680-999dd36

note: this image tag is only available for internal use, please check the internal doc for more details.