search

apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
11.8k stars 3.11k forks source link

[fix](group commit) Fix group commit in nereids #37002

Open mymeiyi opened 3 days ago

mymeiyi commented 3 days ago

Proposed changes

Before: the insert into values in group commit use the stream load way to load data, this may cause complex data type load error. After: use the original insert into values way(scanner node is union node), sink is group_commit_block_sink.

doris-robot commented 3 days ago

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

github-actions[bot] commented 3 days ago

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] commented 3 days ago

clang-tidy review says "All clean, LGTM! :+1:"

mymeiyi commented 3 days ago

run buildall

github-actions[bot] commented 3 days ago

clang-tidy review says "All clean, LGTM! :+1:"

mymeiyi commented 3 days ago

run buildall

github-actions[bot] commented 3 days ago

clang-tidy review says "All clean, LGTM! :+1:"

doris-robot commented 3 days ago
TPC-H: Total hot run time: 40274 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 05e8c24e2e775c3ca1cf1c1af213d6e5852a43ec, data reload: false ------ Round 1 ---------------------------------- q1 18356 4725 4369 4369 q2 2642 198 190 190 q3 11308 1186 1166 1166 q4 11145 831 909 831 q5 7523 2743 2700 2700 q6 219 137 143 137 q7 979 602 617 602 q8 9515 2121 2117 2117 q9 9300 6525 6518 6518 q10 8910 3728 3722 3722 q11 440 240 239 239 q12 402 240 224 224 q13 17786 2961 3030 2961 q14 262 226 211 211 q15 514 478 480 478 q16 505 374 368 368 q17 989 619 636 619 q18 8104 7324 7455 7324 q19 5332 1591 1498 1498 q20 670 324 329 324 q21 4993 3340 3875 3340 q22 400 336 346 336 Total cold run time: 120294 ms Total hot run time: 40274 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4444 4264 4207 4207 q2 376 264 267 264 q3 2954 2693 2735 2693 q4 1912 1585 1639 1585 q5 5271 5295 5313 5295 q6 218 129 129 129 q7 2140 1750 1746 1746 q8 3209 3365 3361 3361 q9 8346 8400 8378 8378 q10 3848 3703 3736 3703 q11 578 494 504 494 q12 786 607 595 595 q13 16400 2999 2990 2990 q14 297 257 259 257 q15 533 486 472 472 q16 457 418 446 418 q17 1787 1499 1478 1478 q18 7543 7440 7380 7380 q19 1772 1584 1549 1549 q20 2020 1778 1777 1777 q21 4845 4743 4835 4743 q22 633 551 552 551 Total cold run time: 70369 ms Total hot run time: 54065 ms ```
doris-robot commented 3 days ago
TPC-DS: Total hot run time: 169385 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 05e8c24e2e775c3ca1cf1c1af213d6e5852a43ec, data reload: false query1 919 384 376 376 query2 6486 2473 2301 2301 query3 6644 209 216 209 query4 19339 17463 17195 17195 query5 4151 491 475 475 query6 273 181 172 172 query7 4605 306 283 283 query8 301 294 304 294 query9 8701 2490 2468 2468 query10 613 307 295 295 query11 10551 9975 10084 9975 query12 131 89 82 82 query13 1636 375 375 375 query14 9753 6973 7640 6973 query15 218 178 190 178 query16 7910 266 259 259 query17 1910 538 519 519 query18 1977 272 265 265 query19 208 145 151 145 query20 85 81 78 78 query21 212 139 128 128 query22 4482 4028 3983 3983 query23 33644 32899 32822 32822 query24 11183 2830 2776 2776 query25 640 359 357 357 query26 1370 144 148 144 query27 2974 312 318 312 query28 7386 2122 2111 2111 query29 909 633 609 609 query30 283 160 153 153 query31 935 729 741 729 query32 92 49 52 49 query33 767 280 305 280 query34 922 475 465 465 query35 739 617 622 617 query36 1088 920 924 920 query37 153 71 68 68 query38 2850 2768 2763 2763 query39 875 783 799 783 query40 202 123 124 123 query41 52 49 48 48 query42 114 100 98 98 query43 620 555 541 541 query44 1246 743 748 743 query45 189 165 166 165 query46 1072 746 706 706 query47 1874 1805 1749 1749 query48 361 295 307 295 query49 1070 404 397 397 query50 763 377 389 377 query51 6836 6911 6679 6679 query52 103 101 91 91 query53 360 284 298 284 query54 899 449 438 438 query55 74 76 75 75 query56 306 263 274 263 query57 1159 1052 1070 1052 query58 253 238 235 235 query59 3486 3315 3209 3209 query60 306 271 271 271 query61 96 91 94 91 query62 660 452 454 452 query63 323 289 294 289 query64 8936 2265 1737 1737 query65 3139 3056 3115 3056 query66 1015 341 327 327 query67 15371 15294 14835 14835 query68 4528 545 542 542 query69 467 297 300 297 query70 1214 1053 1141 1053 query71 394 273 271 271 query72 7054 5546 2781 2781 query73 739 324 321 321 query74 6082 5475 5464 5464 query75 3457 2646 2658 2646 query76 2772 910 940 910 query77 445 303 298 298 query78 10482 9754 9630 9630 query79 2702 516 527 516 query80 1187 468 459 459 query81 599 219 277 219 query82 774 101 102 101 query83 242 168 166 166 query84 240 88 87 87 query85 1670 280 275 275 query86 509 334 327 327 query87 3261 3074 3045 3045 query88 4118 2347 2353 2347 query89 461 391 368 368 query90 1777 190 192 190 query91 126 97 100 97 query92 60 49 48 48 query93 1854 513 521 513 query94 1251 195 186 186 query95 402 308 307 307 query96 592 276 264 264 query97 3238 3017 3088 3017 query98 213 197 190 190 query99 1206 838 861 838 Total cold run time: 272916 ms Total hot run time: 169385 ms ```
doris-robot commented 3 days ago
ClickBench: Total hot run time: 30.48 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 05e8c24e2e775c3ca1cf1c1af213d6e5852a43ec, data reload: false query1 0.04 0.03 0.03 query2 0.08 0.04 0.04 query3 0.23 0.06 0.05 query4 1.68 0.07 0.07 query5 0.50 0.49 0.50 query6 1.14 0.73 0.72 query7 0.02 0.02 0.01 query8 0.05 0.05 0.04 query9 0.55 0.51 0.49 query10 0.55 0.55 0.52 query11 0.16 0.11 0.11 query12 0.16 0.12 0.13 query13 0.59 0.58 0.58 query14 0.77 0.78 0.79 query15 0.84 0.83 0.81 query16 0.37 0.38 0.38 query17 0.99 0.98 0.97 query18 0.22 0.25 0.24 query19 1.77 1.67 1.65 query20 0.01 0.01 0.01 query21 15.44 0.74 0.65 query22 4.34 6.77 2.06 query23 18.32 1.40 1.26 query24 2.08 0.23 0.21 query25 0.16 0.08 0.09 query26 0.26 0.18 0.18 query27 0.08 0.08 0.07 query28 13.29 1.00 1.01 query29 12.63 3.22 3.24 query30 0.25 0.07 0.05 query31 2.87 0.40 0.38 query32 3.27 0.48 0.47 query33 2.88 2.91 2.85 query34 17.26 4.40 4.39 query35 4.48 4.44 4.47 query36 0.65 0.46 0.48 query37 0.18 0.16 0.15 query38 0.16 0.14 0.15 query39 0.05 0.03 0.04 query40 0.19 0.14 0.14 query41 0.10 0.05 0.05 query42 0.06 0.05 0.05 query43 0.04 0.04 0.04 Total cold run time: 109.76 s Total hot run time: 30.48 s ```