search

apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
11.8k stars 3.11k forks source link

[fix](parquet) prevent parquet page reader print much warning logs #37011

Closed AshinGau closed 3 hours ago

AshinGau commented 3 days ago

Proposed changes

Prevent parquet page reader print much warning logs.

Couldn't deserialize thrift msg:
doris-robot commented 3 days ago

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

AshinGau commented 3 days ago

run buidall

github-actions[bot] commented 3 days ago

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] commented 3 days ago

PR approved by at least one committer and no changes requested.

github-actions[bot] commented 3 days ago

PR approved by anyone and no changes requested.

AshinGau commented 1 day ago

run buidall

AshinGau commented 17 hours ago

run buidall

morningman commented 11 hours ago

run buildall

doris-robot commented 9 hours ago
TPC-H: Total hot run time: 39755 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 208b510d9ba95c108cbcd6fe832b7dec4c3f4c8c, data reload: false ------ Round 1 ---------------------------------- q1 17666 4436 4263 4263 q2 2029 195 195 195 q3 10513 1259 1053 1053 q4 10199 798 716 716 q5 7515 2717 2667 2667 q6 224 139 139 139 q7 954 597 594 594 q8 9240 2080 2080 2080 q9 8904 6504 6480 6480 q10 9053 3726 3741 3726 q11 473 244 233 233 q12 501 233 228 228 q13 17779 2974 2984 2974 q14 258 234 221 221 q15 520 484 485 484 q16 504 375 371 371 q17 964 736 641 641 q18 8122 7472 7476 7472 q19 6422 1500 1437 1437 q20 659 330 317 317 q21 4887 3904 3129 3129 q22 394 335 341 335 Total cold run time: 117780 ms Total hot run time: 39755 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4413 4223 4271 4223 q2 376 269 268 268 q3 2948 2887 2855 2855 q4 2008 1681 1768 1681 q5 5580 5487 5450 5450 q6 227 144 130 130 q7 2287 1876 1849 1849 q8 3305 3448 3436 3436 q9 8738 8732 8764 8732 q10 4104 3706 3786 3706 q11 611 503 502 502 q12 827 620 633 620 q13 17311 3143 3171 3143 q14 319 278 268 268 q15 538 489 481 481 q16 490 427 415 415 q17 1801 1543 1495 1495 q18 8195 7959 7813 7813 q19 2107 1747 1625 1625 q20 2923 1905 1866 1866 q21 5098 4816 4654 4654 q22 665 530 583 530 Total cold run time: 74871 ms Total hot run time: 55742 ms ```
doris-robot commented 9 hours ago
TPC-DS: Total hot run time: 173825 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 208b510d9ba95c108cbcd6fe832b7dec4c3f4c8c, data reload: false query1 932 386 379 379 query2 6485 2440 2244 2244 query3 6631 205 216 205 query4 19286 17340 17238 17238 query5 3741 480 459 459 query6 279 162 172 162 query7 4597 301 296 296 query8 322 315 303 303 query9 8738 2394 2399 2394 query10 576 314 284 284 query11 10593 10144 10059 10059 query12 118 86 85 85 query13 1661 370 371 370 query14 10379 7839 7437 7437 query15 243 182 189 182 query16 7825 278 276 276 query17 1878 566 545 545 query18 1675 289 274 274 query19 215 156 158 156 query20 95 85 83 83 query21 220 135 123 123 query22 4445 4063 4024 4024 query23 33842 33511 33784 33511 query24 10835 2970 2895 2895 query25 612 418 389 389 query26 728 160 157 157 query27 2283 326 320 320 query28 5619 2163 2138 2138 query29 891 668 648 648 query30 274 169 167 167 query31 994 757 757 757 query32 99 55 57 55 query33 678 311 297 297 query34 869 485 503 485 query35 756 660 650 650 query36 1127 974 980 974 query37 138 78 81 78 query38 2986 2806 2806 2806 query39 880 858 827 827 query40 216 138 131 131 query41 62 54 62 54 query42 120 105 114 105 query43 594 541 540 540 query44 1086 741 731 731 query45 195 169 173 169 query46 1083 730 712 712 query47 1832 1798 1779 1779 query48 378 307 319 307 query49 891 513 417 417 query50 766 381 382 381 query51 6889 6769 6805 6769 query52 106 96 96 96 query53 363 285 291 285 query54 914 439 450 439 query55 74 75 72 72 query56 292 259 259 259 query57 1108 1028 1043 1028 query58 239 242 259 242 query59 3260 3165 3239 3165 query60 313 287 287 287 query61 89 91 90 90 query62 603 439 443 439 query63 322 283 288 283 query64 8638 2271 1759 1759 query65 3154 3131 3093 3093 query66 736 341 331 331 query67 15681 14855 14908 14855 query68 8632 529 542 529 query69 762 440 351 351 query70 1392 1052 1133 1052 query71 519 283 272 272 query72 8960 5604 5243 5243 query73 2256 319 324 319 query74 5917 5406 5483 5406 query75 4939 2633 2682 2633 query76 5163 1024 930 930 query77 772 321 302 302 query78 10508 9714 9779 9714 query79 8716 510 515 510 query80 1090 465 453 453 query81 547 220 213 213 query82 425 109 103 103 query83 328 170 170 170 query84 275 82 84 82 query85 886 273 274 273 query86 357 296 322 296 query87 3270 3048 3111 3048 query88 4706 2359 2334 2334 query89 532 390 384 384 query90 2047 186 187 186 query91 133 115 133 115 query92 61 47 49 47 query93 6747 502 495 495 query94 1350 189 184 184 query95 412 309 305 305 query96 611 265 259 259 query97 3160 3031 3057 3031 query98 224 203 200 200 query99 1151 836 857 836 Total cold run time: 291397 ms Total hot run time: 173825 ms ```
doris-robot commented 9 hours ago
ClickBench: Total hot run time: 31.01 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 208b510d9ba95c108cbcd6fe832b7dec4c3f4c8c, data reload: false query1 0.04 0.03 0.03 query2 0.07 0.04 0.04 query3 0.22 0.05 0.05 query4 1.68 0.08 0.08 query5 0.51 0.48 0.49 query6 1.12 0.72 0.72 query7 0.02 0.02 0.01 query8 0.05 0.04 0.05 query9 0.55 0.50 0.48 query10 0.55 0.54 0.55 query11 0.15 0.12 0.11 query12 0.14 0.12 0.12 query13 0.59 0.60 0.60 query14 0.77 0.77 0.77 query15 0.85 0.81 0.81 query16 0.36 0.37 0.35 query17 1.00 0.98 1.00 query18 0.22 0.25 0.26 query19 1.78 1.70 1.71 query20 0.01 0.01 0.00 query21 15.43 0.74 0.66 query22 4.39 6.55 2.44 query23 18.29 1.38 1.28 query24 2.10 0.22 0.25 query25 0.17 0.08 0.08 query26 0.25 0.19 0.18 query27 0.08 0.08 0.08 query28 13.19 1.02 0.99 query29 12.68 3.32 3.31 query30 0.26 0.06 0.05 query31 2.88 0.39 0.40 query32 3.26 0.48 0.47 query33 2.91 2.85 2.89 query34 17.19 4.34 4.44 query35 4.47 4.46 4.43 query36 0.65 0.47 0.47 query37 0.19 0.15 0.17 query38 0.16 0.15 0.15 query39 0.04 0.04 0.03 query40 0.17 0.14 0.14 query41 0.09 0.05 0.05 query42 0.06 0.05 0.05 query43 0.04 0.04 0.04 Total cold run time: 109.63 s Total hot run time: 31.01 s ```