[feat](Nereids) push down predicates with multi columns through LogicalWindow and LogicalPartitionTopN

[feiniaofeiafei]

cherry-pick #36828 to branch-2.0

The requirement for predicate pushdown through the window operator is that the partition by slots of the window contains all slots in the predicate. The original implementation of doris only allows predicate pushdown with one slot. This PR relaxes this restriction and allows for predicate pushdown with multiple slots. The same applies to the predicate pushdown of the LogicalPartitionTopN operator. The following sql is an example.

select
    *
from
    (
    select
        row_number() over(partition by id, value1 order by value1) as num,
        id,
        value1
    from
        push_down_multi_column_predicate_through_window_t ) t
where
    abs(id + value1)<4
    and num <= 2;

[doris-robot]

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

[feiniaofeiafei]

run buildall

[feiniaofeiafei]

run buildall

[feiniaofeiafei]

run buildall

[doris-robot]

TPC-H: Total hot run time: 50686 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 9e597ff6e96942efc2a6a31f68d0ff0b6bbf6ad4, data reload: false ------ Round 1 ---------------------------------- q1 17817 4427 4429 4427 q2 2080 160 149 149 q3 10401 1920 1996 1920 q4 10398 1264 1336 1264 q5 9061 3911 3936 3911 q6 259 127 127 127 q7 2034 1640 1637 1637 q8 9371 2811 2724 2724 q9 10834 10348 10550 10348 q10 9008 3674 3712 3674 q11 448 258 271 258 q12 489 317 322 317 q13 19045 4118 4192 4118 q14 374 365 416 365 q15 554 460 464 460 q16 700 602 616 602 q17 1151 983 978 978 q18 7290 6967 6955 6955 q19 2980 1623 1600 1600 q20 957 354 314 314 q21 4506 4116 4087 4087 q22 560 465 451 451 Total cold run time: 120317 ms Total hot run time: 50686 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4349 4316 4322 4316 q2 325 230 231 230 q3 4157 4150 4183 4150 q4 2762 2727 2748 2727 q5 7141 7049 7083 7049 q6 237 122 120 120 q7 3250 2819 2835 2819 q8 4347 4459 4461 4459 q9 16824 16875 16667 16667 q10 4224 4270 4242 4242 q11 787 673 696 673 q12 1040 836 855 836 q13 7003 3741 3740 3740 q14 448 433 425 425 q15 511 463 455 455 q16 741 709 676 676 q17 3826 3828 3882 3828 q18 8842 8746 8868 8746 q19 1719 1698 1658 1658 q20 2361 2161 2112 2112 q21 8621 8459 8466 8459 q22 1029 975 926 926 Total cold run time: 84544 ms Total hot run time: 79313 ms ```

[doris-robot]

TPC-DS: Total hot run time: 204130 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 9e597ff6e96942efc2a6a31f68d0ff0b6bbf6ad4, data reload: false query1 929 420 381 381 query2 6537 2870 2890 2870 query3 6921 207 208 207 query4 20694 17976 17991 17976 query5 19749 6495 6419 6419 query6 285 226 231 226 query7 4155 310 315 310 query8 426 411 401 401 query9 3133 2698 2605 2605 query10 402 310 294 294 query11 11238 10628 10661 10628 query12 116 81 74 74 query13 5595 689 692 689 query14 17763 13801 13341 13341 query15 363 246 255 246 query16 6489 287 260 260 query17 1514 1491 893 893 query18 2271 411 417 411 query19 212 146 156 146 query20 82 78 83 78 query21 193 94 99 94 query22 5186 4983 5003 4983 query23 32574 31888 32005 31888 query24 7022 6556 6499 6499 query25 533 436 428 428 query26 513 169 164 164 query27 1825 300 302 300 query28 6094 2396 2329 2329 query29 2902 2727 2855 2727 query30 242 166 173 166 query31 911 733 735 733 query32 72 68 63 63 query33 388 261 251 251 query34 837 484 487 484 query35 1114 936 917 917 query36 1441 1210 1192 1192 query37 87 61 63 61 query38 3082 2949 2918 2918 query39 1386 1325 1326 1325 query40 214 95 91 91 query41 47 45 44 44 query42 91 82 87 82 query43 724 625 793 625 query44 1153 725 741 725 query45 249 235 237 235 query46 1252 959 967 959 query47 1863 1979 1802 1802 query48 1039 720 717 717 query49 626 384 374 374 query50 863 629 589 589 query51 4755 4651 4568 4568 query52 96 79 83 79 query53 449 335 328 328 query54 2666 2443 2477 2443 query55 87 86 86 86 query56 234 241 218 218 query57 1178 1082 1077 1077 query58 217 213 197 197 query59 4273 4004 4008 4004 query60 212 201 196 196 query61 102 97 100 97 query62 851 448 467 448 query63 492 350 349 349 query64 2237 1533 1417 1417 query65 3632 3553 3564 3553 query66 795 385 385 385 query67 16183 15930 16210 15930 query68 8504 655 690 655 query69 568 375 370 370 query70 1607 1335 1454 1335 query71 381 327 320 320 query72 6649 3530 3526 3526 query73 736 322 329 322 query74 6367 5928 5855 5855 query75 5014 3759 3647 3647 query76 5099 1139 1184 1139 query77 780 257 252 252 query78 12480 11413 20585 11413 query79 6209 642 647 642 query80 1190 411 406 406 query81 500 238 239 238 query82 268 97 102 97 query83 186 146 136 136 query84 260 72 74 72 query85 1022 329 323 323 query86 350 334 311 311 query87 3226 3013 3056 3013 query88 4136 2317 2306 2306 query89 354 289 308 289 query90 1807 211 219 211 query91 190 149 141 141 query92 63 57 53 53 query93 958 550 588 550 query94 731 217 213 213 query95 1132 1105 1075 1075 query96 665 339 335 335 query97 6521 6312 6431 6312 query98 194 173 178 173 query99 3014 805 866 805 Total cold run time: 303017 ms Total hot run time: 204130 ms ```

[doris-robot]

ClickBench: Total hot run time: 30.86 s

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 9e597ff6e96942efc2a6a31f68d0ff0b6bbf6ad4, data reload: false query1 0.02 0.02 0.02 query2 0.07 0.02 0.02 query3 0.25 0.06 0.05 query4 1.77 0.10 0.06 query5 0.54 0.53 0.52 query6 1.28 0.64 0.63 query7 0.02 0.01 0.01 query8 0.03 0.03 0.02 query9 0.53 0.48 0.47 query10 0.54 0.53 0.54 query11 0.11 0.09 0.08 query12 0.12 0.09 0.08 query13 0.63 0.62 0.60 query14 0.78 0.78 0.78 query15 0.79 0.78 0.75 query16 0.37 0.40 0.36 query17 1.05 1.02 0.99 query18 0.22 0.27 0.24 query19 1.94 1.84 1.86 query20 0.01 0.01 0.01 query21 15.47 0.55 0.55 query22 2.09 2.29 1.62 query23 17.06 1.02 0.96 query24 5.25 1.10 1.05 query25 0.40 0.11 0.06 query26 0.57 0.16 0.17 query27 0.03 0.04 0.04 query28 7.71 0.74 0.71 query29 12.58 2.31 2.30 query30 0.61 0.53 0.53 query31 2.82 0.39 0.38 query32 3.38 0.49 0.50 query33 3.03 3.08 3.11 query34 15.26 4.81 4.80 query35 4.87 4.83 4.82 query36 1.06 1.02 1.02 query37 0.06 0.05 0.04 query38 0.03 0.03 0.02 query39 0.02 0.01 0.02 query40 0.16 0.14 0.13 query41 0.06 0.02 0.01 query42 0.02 0.02 0.01 query43 0.03 0.02 0.01 Total cold run time: 103.64 s Total hot run time: 30.86 s ```

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 9e597ff6e96942efc2a6a31f68d0ff0b6bbf6ad4 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

[feiniaofeiafei]

run buildall

[doris-robot]

TPC-H: Total hot run time: 49843 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 91dffca63df8d99da45e856f264bf811cd07a40b, data reload: false ------ Round 1 ---------------------------------- q1 17542 4451 4335 4335 q2 2078 153 146 146 q3 10468 1882 1967 1882 q4 10312 1226 1336 1226 q5 8409 3875 3925 3875 q6 233 127 125 125 q7 2073 1608 1600 1600 q8 9520 2727 2710 2710 q9 13883 11079 10323 10323 q10 8673 3520 3538 3520 q11 422 247 238 238 q12 487 309 309 309 q13 18348 3946 4062 3946 q14 356 329 335 329 q15 497 452 458 452 q16 673 579 571 571 q17 1128 981 945 945 q18 7304 6856 6826 6826 q19 1780 1635 1621 1621 q20 570 315 315 315 q21 4443 4112 4117 4112 q22 531 463 437 437 Total cold run time: 119730 ms Total hot run time: 49843 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4370 4314 4352 4314 q2 319 230 216 216 q3 4156 4175 4175 4175 q4 2753 2752 2770 2752 q5 7157 7121 7071 7071 q6 239 118 121 118 q7 3208 2841 2786 2786 q8 4366 4442 4443 4442 q9 16912 16778 16898 16778 q10 4270 4242 4283 4242 q11 747 706 709 706 q12 1031 846 877 846 q13 7012 3742 3769 3742 q14 470 426 434 426 q15 504 471 461 461 q16 748 690 698 690 q17 3846 3913 3820 3820 q18 8888 8709 8799 8709 q19 1730 1693 1702 1693 q20 2386 2122 2084 2084 q21 8479 8375 8451 8375 q22 1036 993 1004 993 Total cold run time: 84627 ms Total hot run time: 79439 ms ```

[doris-robot]

TPC-DS: Total hot run time: 203647 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 91dffca63df8d99da45e856f264bf811cd07a40b, data reload: false query1 960 387 380 380 query2 6535 2855 2822 2822 query3 6915 206 203 203 query4 19869 18109 17894 17894 query5 19728 6530 6553 6530 query6 294 223 231 223 query7 4149 292 305 292 query8 449 397 452 397 query9 3139 2698 2599 2599 query10 409 306 308 306 query11 11322 10677 10786 10677 query12 128 80 80 80 query13 5595 692 695 692 query14 17798 13724 13735 13724 query15 357 250 246 246 query16 6458 305 260 260 query17 1688 1453 868 868 query18 2312 415 404 404 query19 223 149 152 149 query20 86 78 79 78 query21 192 92 98 92 query22 5229 4891 4953 4891 query23 32739 32149 32017 32017 query24 6972 6556 6505 6505 query25 522 436 440 436 query26 620 160 163 160 query27 1998 300 291 291 query28 6121 2346 2298 2298 query29 2961 2638 2772 2638 query30 242 169 168 168 query31 889 720 733 720 query32 62 63 62 62 query33 394 262 252 252 query34 835 474 479 474 query35 1116 924 910 910 query36 1310 1135 1181 1135 query37 90 59 60 59 query38 3065 2912 2977 2912 query39 1366 1346 1306 1306 query40 219 95 92 92 query41 47 44 44 44 query42 91 83 76 76 query43 814 723 779 723 query44 1147 715 706 706 query45 251 238 232 232 query46 1215 980 957 957 query47 1829 1750 1686 1686 query48 1004 721 715 715 query49 625 382 365 365 query50 863 617 601 601 query51 4687 4639 4685 4639 query52 113 79 104 79 query53 440 323 334 323 query54 2697 2449 2536 2449 query55 80 79 84 79 query56 226 206 208 206 query57 1233 1149 1193 1149 query58 218 214 192 192 query59 4264 4225 3935 3935 query60 210 213 209 209 query61 95 94 93 93 query62 813 474 418 418 query63 495 339 342 339 query64 2638 1547 1352 1352 query65 3609 3557 3554 3554 query66 795 376 371 371 query67 16379 16630 15232 15232 query68 8378 646 641 641 query69 552 348 341 341 query70 1606 1353 1461 1353 query71 405 308 320 308 query72 6501 3535 3464 3464 query73 723 318 317 317 query74 6219 5840 5846 5840 query75 5065 3822 3655 3655 query76 5079 1166 1210 1166 query77 859 261 252 252 query78 12500 11652 11746 11652 query79 7658 630 635 630 query80 1029 399 405 399 query81 490 234 233 233 query82 1491 97 99 97 query83 167 130 131 130 query84 262 70 70 70 query85 878 313 317 313 query86 344 298 291 291 query87 3249 3073 3025 3025 query88 5032 2329 2322 2322 query89 360 307 292 292 query90 1953 188 205 188 query91 183 139 139 139 query92 60 52 51 51 query93 4781 564 579 564 query94 729 214 206 206 query95 1102 1064 1027 1027 query96 638 319 325 319 query97 6513 6390 6385 6385 query98 197 176 186 176 query99 2836 835 879 835 Total cold run time: 310073 ms Total hot run time: 203647 ms ```

[doris-robot]

ClickBench: Total hot run time: 30.84 s

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 91dffca63df8d99da45e856f264bf811cd07a40b, data reload: false query1 0.02 0.02 0.03 query2 0.06 0.03 0.03 query3 0.24 0.05 0.05 query4 1.79 0.06 0.06 query5 0.54 0.52 0.52 query6 1.23 0.67 0.61 query7 0.01 0.01 0.01 query8 0.03 0.02 0.02 query9 0.53 0.49 0.49 query10 0.54 0.53 0.54 query11 0.12 0.09 0.09 query12 0.12 0.09 0.09 query13 0.62 0.61 0.61 query14 0.78 0.77 0.79 query15 0.77 0.77 0.76 query16 0.36 0.35 0.35 query17 1.02 1.03 1.04 query18 0.24 0.25 0.26 query19 1.93 1.89 1.84 query20 0.01 0.01 0.01 query21 15.46 0.56 0.55 query22 2.23 2.01 1.54 query23 17.20 1.05 0.87 query24 5.88 1.26 1.31 query25 0.40 0.09 0.06 query26 0.66 0.16 0.16 query27 0.05 0.04 0.03 query28 6.83 0.72 0.71 query29 12.60 2.36 2.32 query30 0.52 0.53 0.46 query31 2.82 0.39 0.37 query32 3.38 0.50 0.49 query33 3.07 3.04 3.04 query34 15.26 4.78 4.77 query35 4.84 4.82 4.85 query36 1.09 1.02 1.01 query37 0.05 0.04 0.05 query38 0.03 0.02 0.02 query39 0.02 0.01 0.02 query40 0.16 0.14 0.13 query41 0.07 0.01 0.01 query42 0.02 0.02 0.01 query43 0.02 0.02 0.02 Total cold run time: 103.62 s Total hot run time: 30.84 s ```

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 91dffca63df8d99da45e856f264bf811cd07a40b with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.1 seconds inserted 10000000 Rows, about 473K ops/s

[feiniaofeiafei]

run buildall

[doris-robot]

TPC-H: Total hot run time: 50028 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 33cfd3d5a54ee04864c9a1ef8796bddba0acb422, data reload: false ------ Round 1 ---------------------------------- q1 17782 4396 4341 4341 q2 2064 153 146 146 q3 10279 1973 1885 1885 q4 10351 1231 1311 1231 q5 8417 3896 3886 3886 q6 227 152 126 126 q7 2020 1592 1624 1592 q8 9490 2698 2698 2698 q9 12686 10523 10533 10523 q10 8633 3509 3502 3502 q11 418 259 254 254 q12 471 304 301 301 q13 18372 3948 4049 3948 q14 362 329 324 324 q15 496 457 447 447 q16 691 561 570 561 q17 1115 965 911 911 q18 7263 6994 6957 6957 q19 1782 1661 1598 1598 q20 529 296 311 296 q21 4336 4160 4063 4063 q22 536 438 447 438 Total cold run time: 118320 ms Total hot run time: 50028 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4331 4361 4298 4298 q2 318 232 234 232 q3 4171 4143 4120 4120 q4 2744 2751 2741 2741 q5 7112 7058 7072 7058 q6 238 117 118 117 q7 3192 2853 2843 2843 q8 4306 4414 4434 4414 q9 17353 16870 16977 16870 q10 4206 4247 4278 4247 q11 734 694 708 694 q12 1035 855 840 840 q13 6667 3737 3773 3737 q14 472 437 420 420 q15 514 462 458 458 q16 725 683 684 683 q17 3787 3836 3784 3784 q18 8847 8753 8931 8753 q19 1730 1716 1690 1690 q20 2374 2098 2113 2098 q21 8394 8488 8384 8384 q22 1048 970 984 970 Total cold run time: 84298 ms Total hot run time: 79451 ms ```

[doris-robot]

TPC-DS: Total hot run time: 203289 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 33cfd3d5a54ee04864c9a1ef8796bddba0acb422, data reload: false query1 922 426 384 384 query2 6543 2845 2508 2508 query3 6921 207 206 206 query4 20091 18145 18061 18061 query5 19727 6550 6508 6508 query6 303 226 238 226 query7 4226 300 299 299 query8 390 392 369 369 query9 3103 2686 2610 2610 query10 416 299 314 299 query11 11344 10733 10824 10733 query12 130 79 74 74 query13 5596 692 695 692 query14 17866 13319 13245 13245 query15 368 246 246 246 query16 6513 289 265 265 query17 1698 1467 914 914 query18 3369 426 409 409 query19 487 159 149 149 query20 78 76 80 76 query21 192 106 94 94 query22 5247 5028 5092 5028 query23 32902 32690 32481 32481 query24 7472 6549 6614 6549 query25 564 427 449 427 query26 513 171 162 162 query27 2069 305 298 298 query28 8592 2303 2275 2275 query29 2819 2669 2650 2650 query30 244 168 167 167 query31 932 744 747 744 query32 72 67 64 64 query33 398 287 262 262 query34 845 466 473 466 query35 1141 982 913 913 query36 1223 1113 1277 1113 query37 93 61 63 61 query38 3031 2939 2941 2939 query39 1373 1335 1331 1331 query40 203 93 101 93 query41 47 43 45 43 query42 86 80 85 80 query43 733 662 733 662 query44 1115 714 721 714 query45 249 249 241 241 query46 1231 961 978 961 query47 1794 1825 1808 1808 query48 1021 689 704 689 query49 617 379 374 374 query50 855 620 613 613 query51 4794 4640 4610 4610 query52 96 81 79 79 query53 450 326 321 321 query54 2658 2470 2474 2470 query55 86 77 83 77 query56 238 221 215 215 query57 1313 1178 1128 1128 query58 223 211 182 182 query59 4110 4072 3722 3722 query60 210 196 206 196 query61 101 96 98 96 query62 789 483 504 483 query63 491 339 335 335 query64 2526 1552 1425 1425 query65 3633 3527 3541 3527 query66 775 381 381 381 query67 16060 15743 14781 14781 query68 9053 663 693 663 query69 569 341 357 341 query70 1582 1437 1655 1437 query71 432 305 325 305 query72 6545 3527 3472 3472 query73 732 319 315 315 query74 6238 5865 5905 5865 query75 5378 3594 3629 3594 query76 5608 1172 1077 1077 query77 953 256 260 256 query78 12713 11848 11567 11567 query79 8050 623 640 623 query80 1231 407 407 407 query81 493 237 234 234 query82 1658 101 97 97 query83 178 135 136 135 query84 260 71 71 71 query85 881 325 332 325 query86 332 299 337 299 query87 3285 3026 3054 3026 query88 5198 2361 2350 2350 query89 384 316 292 292 query90 1962 211 205 205 query91 170 139 138 138 query92 57 53 54 53 query93 5484 573 586 573 query94 706 213 210 210 query95 1114 1060 1042 1042 query96 649 333 330 330 query97 6531 6378 6525 6378 query98 188 175 179 175 query99 2917 883 831 831 Total cold run time: 317848 ms Total hot run time: 203289 ms ```

[doris-robot]

ClickBench: Total hot run time: 30.62 s

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 33cfd3d5a54ee04864c9a1ef8796bddba0acb422, data reload: false query1 0.02 0.02 0.03 query2 0.07 0.02 0.02 query3 0.25 0.05 0.05 query4 1.78 0.07 0.06 query5 0.54 0.52 0.51 query6 1.25 0.61 0.61 query7 0.01 0.01 0.01 query8 0.04 0.02 0.02 query9 0.52 0.49 0.48 query10 0.54 0.54 0.54 query11 0.12 0.09 0.08 query12 0.12 0.09 0.09 query13 0.62 0.62 0.62 query14 0.78 0.77 0.79 query15 0.77 0.75 0.76 query16 0.37 0.39 0.39 query17 0.95 0.99 1.02 query18 0.21 0.28 0.22 query19 1.96 1.82 1.85 query20 0.02 0.01 0.01 query21 15.46 0.57 0.56 query22 2.04 1.95 1.42 query23 16.67 1.11 0.85 query24 5.48 1.24 1.34 query25 0.41 0.07 0.05 query26 0.62 0.16 0.17 query27 0.04 0.03 0.04 query28 7.06 0.74 0.76 query29 12.63 2.36 2.22 query30 0.60 0.53 0.54 query31 2.81 0.38 0.38 query32 3.39 0.51 0.50 query33 3.04 2.99 3.06 query34 15.24 4.83 4.80 query35 4.88 4.86 4.85 query36 1.09 1.01 1.00 query37 0.05 0.04 0.04 query38 0.03 0.02 0.02 query39 0.02 0.01 0.02 query40 0.16 0.13 0.14 query41 0.06 0.01 0.02 query42 0.02 0.01 0.02 query43 0.02 0.02 0.02 Total cold run time: 102.76 s Total hot run time: 30.62 s ```

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 33cfd3d5a54ee04864c9a1ef8796bddba0acb422 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

[feiniaofeiafei]

run buildall

[doris-robot]

TPC-H: Total hot run time: 50266 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 37a204165305766aeaa64fdca3736bbeac53ed54, data reload: false ------ Round 1 ---------------------------------- q1 17823 4386 4376 4376 q2 2079 160 154 154 q3 10256 1945 1937 1937 q4 10106 1291 1292 1291 q5 8408 3956 3987 3956 q6 256 129 127 127 q7 2015 1623 1609 1609 q8 9302 2748 2724 2724 q9 11161 10727 10359 10359 q10 8656 3525 3478 3478 q11 422 242 245 242 q12 477 310 308 308 q13 18369 3956 4042 3956 q14 367 326 328 326 q15 503 462 459 459 q16 690 579 569 569 q17 1125 965 965 965 q18 7355 6978 6962 6962 q19 1790 1651 1629 1629 q20 572 317 309 309 q21 4437 4104 4093 4093 q22 530 442 437 437 Total cold run time: 116699 ms Total hot run time: 50266 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4343 4308 4308 4308 q2 328 235 222 222 q3 4228 4204 4150 4150 q4 2750 2745 2739 2739 q5 7245 7262 7196 7196 q6 242 124 123 123 q7 3250 2876 2855 2855 q8 4341 4448 4456 4448 q9 17441 17215 17137 17137 q10 4210 4298 4265 4265 q11 767 713 691 691 q12 1030 858 878 858 q13 6886 3738 3723 3723 q14 454 437 430 430 q15 518 456 464 456 q16 739 687 668 668 q17 3924 3921 3842 3842 q18 8905 8795 8763 8763 q19 1718 1710 1687 1687 q20 2384 2154 2100 2100 q21 8422 8391 8538 8391 q22 1069 975 956 956 Total cold run time: 85194 ms Total hot run time: 80008 ms ```

[doris-robot]

TPC-DS: Total hot run time: 203871 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 37a204165305766aeaa64fdca3736bbeac53ed54, data reload: false query1 929 396 410 396 query2 6527 2829 2652 2652 query3 6923 206 207 206 query4 20263 17870 17902 17870 query5 19724 6497 6535 6497 query6 309 224 228 224 query7 4159 302 314 302 query8 423 400 385 385 query9 3146 2724 2660 2660 query10 434 304 310 304 query11 11413 10745 10758 10745 query12 125 75 76 75 query13 5611 699 693 693 query14 17895 13434 13188 13188 query15 358 245 246 245 query16 6459 297 266 266 query17 1693 1465 884 884 query18 2295 420 412 412 query19 203 152 149 149 query20 75 81 81 81 query21 189 91 95 91 query22 5072 5004 4954 4954 query23 32759 31964 31933 31933 query24 7033 6489 6533 6489 query25 540 447 410 410 query26 539 163 159 159 query27 1876 294 293 293 query28 6140 2360 2328 2328 query29 2868 2844 2590 2590 query30 241 167 170 167 query31 902 738 762 738 query32 68 66 60 60 query33 406 262 255 255 query34 841 468 479 468 query35 1159 955 960 955 query36 1351 1281 1134 1134 query37 90 59 60 59 query38 3077 2966 2936 2936 query39 1364 1362 1319 1319 query40 204 94 98 94 query41 44 44 43 43 query42 88 94 81 81 query43 716 631 650 631 query44 1122 711 717 711 query45 249 236 233 233 query46 1225 966 986 966 query47 1756 1853 1694 1694 query48 1011 728 697 697 query49 613 383 380 380 query50 849 629 625 625 query51 4817 4625 4688 4625 query52 106 80 91 80 query53 451 320 315 315 query54 2652 2522 2455 2455 query55 99 74 80 74 query56 209 221 215 215 query57 1091 1141 1089 1089 query58 218 207 193 193 query59 4263 4030 4183 4030 query60 210 219 214 214 query61 100 94 96 94 query62 902 548 577 548 query63 485 345 343 343 query64 2445 1581 1353 1353 query65 3659 3571 3553 3553 query66 821 368 392 368 query67 19023 15614 15773 15614 query68 7957 658 642 642 query69 573 349 343 343 query70 1724 1422 1411 1411 query71 406 298 313 298 query72 6525 3546 3534 3534 query73 740 321 328 321 query74 6360 5837 5852 5837 query75 4679 3750 3692 3692 query76 4558 1137 1214 1137 query77 546 259 272 259 query78 12743 11968 11688 11688 query79 8725 644 638 638 query80 3055 420 407 407 query81 525 233 230 230 query82 1615 97 97 97 query83 201 135 131 131 query84 256 72 71 71 query85 1443 324 321 321 query86 371 290 296 290 query87 3240 3006 3042 3006 query88 5336 2344 2313 2313 query89 383 281 272 272 query90 1726 212 223 212 query91 170 138 137 137 query92 58 52 55 52 query93 5030 559 555 555 query94 899 214 219 214 query95 1094 1060 1033 1033 query96 652 316 326 316 query97 6598 6481 6484 6481 query98 202 185 175 175 query99 2891 868 884 868 Total cold run time: 316188 ms Total hot run time: 203871 ms ```

[doris-robot]

ClickBench: Total hot run time: 30.08 s

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 37a204165305766aeaa64fdca3736bbeac53ed54, data reload: false query1 0.02 0.02 0.02 query2 0.08 0.02 0.02 query3 0.25 0.05 0.05 query4 1.78 0.06 0.07 query5 0.55 0.52 0.51 query6 1.25 0.62 0.61 query7 0.02 0.01 0.01 query8 0.03 0.02 0.02 query9 0.54 0.49 0.48 query10 0.54 0.52 0.54 query11 0.12 0.08 0.08 query12 0.11 0.09 0.09 query13 0.62 0.62 0.61 query14 0.79 0.79 0.78 query15 0.78 0.75 0.77 query16 0.36 0.37 0.38 query17 1.02 0.99 1.01 query18 0.22 0.26 0.25 query19 1.94 1.85 1.89 query20 0.02 0.01 0.01 query21 15.45 0.56 0.54 query22 2.01 2.26 1.43 query23 17.22 0.93 0.87 query24 8.13 0.83 0.65 query25 0.35 0.09 0.05 query26 0.78 0.15 0.16 query27 0.04 0.04 0.04 query28 5.64 0.80 0.77 query29 12.65 2.22 2.26 query30 0.55 0.50 0.48 query31 2.81 0.39 0.37 query32 3.39 0.50 0.49 query33 3.09 3.07 3.02 query34 15.26 4.85 4.80 query35 4.90 4.86 4.89 query36 1.07 1.02 1.01 query37 0.05 0.05 0.04 query38 0.03 0.02 0.02 query39 0.02 0.01 0.02 query40 0.16 0.14 0.14 query41 0.06 0.02 0.01 query42 0.02 0.01 0.02 query43 0.02 0.02 0.02 Total cold run time: 104.74 s Total hot run time: 30.08 s ```

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 37a204165305766aeaa64fdca3736bbeac53ed54 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.7 seconds inserted 10000000 Rows, about 460K ops/s