search

apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
11.8k stars 3.11k forks source link

[fix](inverted index)Make build index operation only affect base index (#36869) #36991

Closed qidaye closed 3 days ago

qidaye commented 3 days ago

backport #36869

doris-robot commented 3 days ago

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

qidaye commented 3 days ago

run buildall

doris-robot commented 3 days ago
TPC-H: Total hot run time: 49649 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 1b6e8de0b6ced398936a6e1e4f3f0f37ddada88b, data reload: false ------ Round 1 ---------------------------------- q1 17803 4442 4348 4348 q2 2069 156 149 149 q3 10301 1932 1952 1932 q4 10364 1250 1357 1250 q5 8472 3900 3907 3900 q6 232 125 126 125 q7 2050 1617 1625 1617 q8 9547 2744 2698 2698 q9 14065 10294 10117 10117 q10 8628 3521 3494 3494 q11 413 255 249 249 q12 480 301 296 296 q13 18344 3913 4001 3913 q14 367 320 339 320 q15 502 466 458 458 q16 680 565 568 565 q17 1132 937 907 907 q18 7250 6932 6995 6932 q19 1778 1648 1558 1558 q20 509 311 302 302 q21 4465 4129 4075 4075 q22 523 459 444 444 Total cold run time: 119974 ms Total hot run time: 49649 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4317 4291 4270 4270 q2 318 234 227 227 q3 4164 4152 4128 4128 q4 2738 2752 2755 2752 q5 7136 7113 7113 7113 q6 240 120 117 117 q7 3271 2887 2847 2847 q8 4363 4467 4472 4467 q9 16908 16713 16788 16713 q10 4198 4221 4295 4221 q11 744 685 697 685 q12 1045 853 868 853 q13 7186 3718 3730 3718 q14 446 430 420 420 q15 506 460 458 458 q16 743 683 673 673 q17 3839 3893 3834 3834 q18 8821 8679 8734 8679 q19 1743 1672 1689 1672 q20 2378 2153 2123 2123 q21 8524 8565 8507 8507 q22 1058 987 954 954 Total cold run time: 84686 ms Total hot run time: 79431 ms ```
doris-robot commented 3 days ago
TPC-DS: Total hot run time: 203239 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 1b6e8de0b6ced398936a6e1e4f3f0f37ddada88b, data reload: false query1 935 423 381 381 query2 6528 2858 2688 2688 query3 6921 208 205 205 query4 20102 18047 17960 17960 query5 19728 6521 6573 6521 query6 295 217 234 217 query7 4151 302 298 298 query8 498 394 435 394 query9 3137 2669 2609 2609 query10 430 285 299 285 query11 11351 10579 10707 10579 query12 133 80 72 72 query13 5595 691 676 676 query14 17913 13237 13220 13220 query15 353 240 256 240 query16 6478 288 271 271 query17 1508 1489 878 878 query18 2275 425 412 412 query19 211 148 150 148 query20 79 81 82 81 query21 190 95 93 93 query22 5330 5086 5106 5086 query23 32537 31810 32213 31810 query24 6961 6516 6527 6516 query25 540 436 427 427 query26 620 168 165 165 query27 1993 307 295 295 query28 6186 2385 2322 2322 query29 2922 2852 2665 2665 query30 243 170 171 170 query31 919 730 763 730 query32 66 66 61 61 query33 411 268 255 255 query34 848 478 467 467 query35 1128 907 864 864 query36 1304 1204 1199 1199 query37 88 58 59 58 query38 3075 2903 2933 2903 query39 1390 1319 1301 1301 query40 208 90 94 90 query41 46 46 44 44 query42 88 82 86 82 query43 679 631 771 631 query44 1165 721 733 721 query45 249 233 238 233 query46 1217 966 954 954 query47 1805 1742 1670 1670 query48 1026 714 691 691 query49 616 382 362 362 query50 863 606 598 598 query51 4782 4639 4617 4617 query52 103 79 82 79 query53 445 329 311 311 query54 2659 2458 2456 2456 query55 83 82 81 81 query56 238 212 219 212 query57 1262 1076 1163 1076 query58 228 188 181 181 query59 4377 4027 3766 3766 query60 213 192 199 192 query61 102 97 97 97 query62 795 537 459 459 query63 485 339 335 335 query64 2608 1523 1532 1523 query65 3616 3547 3583 3547 query66 826 391 389 389 query67 15815 16756 15812 15812 query68 8444 654 684 654 query69 570 382 361 361 query70 1576 1337 1582 1337 query71 405 316 317 316 query72 6548 3518 3492 3492 query73 739 313 307 307 query74 6268 5782 5774 5774 query75 5032 3727 3679 3679 query76 5091 1125 1179 1125 query77 736 257 248 248 query78 12563 11642 11553 11553 query79 9151 637 652 637 query80 1511 408 412 408 query81 507 231 237 231 query82 1685 97 99 97 query83 176 133 132 132 query84 259 72 72 72 query85 877 322 317 317 query86 339 309 318 309 query87 3194 2974 3037 2974 query88 5034 2328 2313 2313 query89 404 301 290 290 query90 1925 213 210 210 query91 195 143 141 141 query92 60 54 54 54 query93 6250 579 553 553 query94 727 208 209 208 query95 1141 1074 1044 1044 query96 642 332 327 327 query97 6433 6375 6365 6365 query98 186 173 176 173 query99 2954 807 810 807 Total cold run time: 313593 ms Total hot run time: 203239 ms ```
doris-robot commented 3 days ago
ClickBench: Total hot run time: 30.8 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 1b6e8de0b6ced398936a6e1e4f3f0f37ddada88b, data reload: false query1 0.02 0.02 0.02 query2 0.07 0.03 0.03 query3 0.25 0.05 0.06 query4 1.77 0.07 0.06 query5 0.56 0.52 0.53 query6 1.28 0.61 0.62 query7 0.01 0.01 0.02 query8 0.03 0.02 0.02 query9 0.55 0.48 0.49 query10 0.53 0.53 0.52 query11 0.12 0.09 0.08 query12 0.12 0.10 0.09 query13 0.62 0.61 0.61 query14 0.77 0.79 0.80 query15 0.77 0.77 0.76 query16 0.35 0.38 0.36 query17 1.04 0.98 1.01 query18 0.21 0.27 0.24 query19 1.94 1.84 1.84 query20 0.01 0.01 0.01 query21 15.49 0.56 0.57 query22 1.96 2.03 1.71 query23 17.11 1.16 0.99 query24 5.87 1.60 0.94 query25 0.38 0.07 0.07 query26 0.71 0.15 0.15 query27 0.04 0.04 0.03 query28 7.03 0.77 0.71 query29 12.65 2.23 2.31 query30 0.63 0.58 0.52 query31 2.81 0.40 0.38 query32 3.36 0.49 0.50 query33 3.09 3.08 3.07 query34 15.26 4.80 4.80 query35 4.87 4.86 4.82 query36 1.07 1.02 1.01 query37 0.06 0.04 0.05 query38 0.03 0.02 0.02 query39 0.02 0.01 0.02 query40 0.16 0.14 0.14 query41 0.07 0.01 0.01 query42 0.02 0.01 0.01 query43 0.03 0.01 0.02 Total cold run time: 103.74 s Total hot run time: 30.8 s ```
doris-robot commented 3 days ago

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1b6e8de0b6ced398936a6e1e4f3f0f37ddada88b with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.1 seconds inserted 10000000 Rows, about 473K ops/s
qidaye commented 3 days ago

run external