[fix](mtmv)fix mtmv dead lock

zddr commented 3 days ago

Cause of occurrence：

dropBaseTable: Holding db's writeLock ,notify mtmv alter status to schema_change,need mv's writeLock
task(insert overwrite):Holding mv's readLock,when generage plan need db's readLock

fix:

mtmv alter status to schema_change need mv's writeMvLock instead of mv's writeLock

doris-robot commented 3 days ago

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

zddr commented 3 days ago

run buildall

doris-robot commented 3 days ago

TPC-H: Total hot run time: 39403 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit fd6681415945a3fcfc7b707f5726f9c187aa9d2d, data reload: false ------ Round 1 ---------------------------------- q1 17616 4270 4236 4236 q2 2012 191 193 191 q3 10494 1138 992 992 q4 10182 763 752 752 q5 7461 2632 2611 2611 q6 216 132 135 132 q7 945 582 599 582 q8 9226 2036 2064 2036 q9 8857 6460 6413 6413 q10 9020 3693 3648 3648 q11 433 239 233 233 q12 479 232 225 225 q13 17770 2997 2958 2958 q14 257 228 221 221 q15 525 481 480 480 q16 514 377 371 371 q17 954 697 710 697 q18 7854 7339 7411 7339 q19 1904 1451 1540 1451 q20 643 318 312 312 q21 4888 3179 3895 3179 q22 392 348 344 344 Total cold run time: 112642 ms Total hot run time: 39403 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4320 4182 4166 4166 q2 373 269 268 268 q3 2931 2772 2649 2649 q4 1903 1685 1727 1685 q5 5539 5565 5543 5543 q6 238 132 131 131 q7 2244 1869 1797 1797 q8 3268 3397 3364 3364 q9 8667 8673 8645 8645 q10 4114 3856 3918 3856 q11 577 488 475 475 q12 748 584 603 584 q13 17095 3184 3124 3124 q14 309 265 276 265 q15 527 495 513 495 q16 490 427 430 427 q17 1793 1526 1496 1496 q18 8076 7996 7744 7744 q19 5933 1665 1541 1541 q20 2052 1842 1900 1842 q21 5085 4846 4927 4846 q22 610 551 558 551 Total cold run time: 76892 ms Total hot run time: 55494 ms ```

doris-robot commented 3 days ago

TPC-DS: Total hot run time: 172832 ms

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit fd6681415945a3fcfc7b707f5726f9c187aa9d2d, data reload: false query1 933 374 377 374 query2 6453 2503 2268 2268 query3 6634 208 215 208 query4 19365 17680 17239 17239 query5 3613 477 492 477 query6 276 160 165 160 query7 4581 293 286 286 query8 324 301 282 282 query9 8644 2449 2446 2446 query10 559 284 273 273 query11 10574 10062 9983 9983 query12 113 81 80 80 query13 1633 374 376 374 query14 10103 6250 7002 6250 query15 233 190 184 184 query16 7622 264 259 259 query17 1705 521 552 521 query18 1682 270 265 265 query19 189 153 152 152 query20 86 80 82 80 query21 215 132 129 129 query22 4418 4086 4042 4042 query23 34052 33372 33603 33372 query24 10831 2843 2815 2815 query25 588 379 369 369 query26 708 162 163 162 query27 2274 338 329 329 query28 6191 2192 2167 2167 query29 875 619 626 619 query30 256 173 156 156 query31 987 785 756 756 query32 98 54 57 54 query33 680 277 288 277 query34 865 482 479 479 query35 762 640 651 640 query36 1093 993 981 981 query37 141 85 75 75 query38 2966 2853 2817 2817 query39 883 839 832 832 query40 217 130 137 130 query41 62 55 59 55 query42 110 102 107 102 query43 589 553 547 547 query44 1123 754 740 740 query45 192 165 167 165 query46 1060 709 708 708 query47 1840 1766 1736 1736 query48 360 308 298 298 query49 824 390 411 390 query50 755 385 380 380 query51 6863 6851 6728 6728 query52 110 95 98 95 query53 364 301 305 301 query54 856 448 434 434 query55 76 75 79 75 query56 289 258 252 252 query57 1137 1047 1040 1040 query58 243 243 251 243 query59 3375 3260 3417 3260 query60 294 271 266 266 query61 95 96 92 92 query62 614 445 445 445 query63 323 293 289 289 query64 8499 2232 1730 1730 query65 3185 3098 3071 3071 query66 737 320 326 320 query67 15292 14998 14752 14752 query68 5802 554 555 554 query69 629 471 393 393 query70 1204 1154 1089 1089 query71 429 272 268 268 query72 7338 5895 5403 5403 query73 773 334 324 324 query74 5947 5544 5493 5493 query75 3606 2696 2651 2651 query76 3161 1002 923 923 query77 608 301 313 301 query78 10261 9860 9774 9774 query79 2204 511 529 511 query80 937 486 482 482 query81 545 222 224 222 query82 1454 111 106 106 query83 269 186 179 179 query84 237 91 87 87 query85 1333 344 328 328 query86 470 327 320 320 query87 3257 3063 3065 3063 query88 3829 2401 2403 2401 query89 472 395 393 393 query90 1713 199 198 198 query91 144 108 111 108 query92 59 51 53 51 query93 2260 525 519 519 query94 1065 193 198 193 query95 421 321 324 321 query96 594 267 265 265 query97 3220 3027 3002 3002 query98 223 204 203 203 query99 1192 831 833 831 Total cold run time: 269345 ms Total hot run time: 172832 ms ```

github-actions[bot] commented 3 days ago

PR approved by at least one committer and no changes requested.

github-actions[bot] commented 3 days ago

PR approved by anyone and no changes requested.

doris-robot commented 3 days ago

ClickBench: Total hot run time: 30.44 s

``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit fd6681415945a3fcfc7b707f5726f9c187aa9d2d, data reload: false query1 0.04 0.04 0.04 query2 0.08 0.04 0.04 query3 0.22 0.06 0.05 query4 1.67 0.07 0.07 query5 0.49 0.50 0.51 query6 1.14 0.73 0.72 query7 0.02 0.01 0.02 query8 0.05 0.04 0.04 query9 0.55 0.50 0.49 query10 0.55 0.55 0.54 query11 0.15 0.12 0.12 query12 0.15 0.12 0.12 query13 0.60 0.59 0.60 query14 0.76 0.76 0.79 query15 0.84 0.81 0.81 query16 0.36 0.37 0.37 query17 0.97 1.01 1.02 query18 0.22 0.24 0.25 query19 1.77 1.72 1.69 query20 0.01 0.00 0.00 query21 15.42 0.72 0.65 query22 4.54 6.83 1.79 query23 18.31 1.38 1.30 query24 2.07 0.21 0.23 query25 0.15 0.08 0.08 query26 0.26 0.17 0.17 query27 0.08 0.08 0.08 query28 13.31 1.02 0.99 query29 12.64 3.24 3.26 query30 0.26 0.06 0.05 query31 2.88 0.38 0.38 query32 3.28 0.47 0.48 query33 2.85 2.93 2.89 query34 16.95 4.43 4.43 query35 4.48 4.50 4.46 query36 0.65 0.46 0.48 query37 0.18 0.16 0.15 query38 0.15 0.15 0.15 query39 0.04 0.04 0.03 query40 0.17 0.13 0.14 query41 0.09 0.05 0.04 query42 0.05 0.05 0.04 query43 0.04 0.04 0.04 Total cold run time: 109.49 s Total hot run time: 30.44 s ```

apache / doris

[fix](mtmv)fix mtmv dead lock #37009