apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.77k stars 3.29k forks source link

[Bug](function)fix json_object function check null nums error as return bool #44321

Closed zhangstar333 closed 1 day ago

zhangstar333 commented 2 days ago

What problem does this PR solve?

Related PR: #https://github.com/apache/doris/pull/34591/files

Problem Summary: before the SQL report error, as it error change the simd::count_zero_num function return value static as bool, so it run into the if check of (not_null_num < size). but actual it's maybe return int type.

now:

mysql [(none)]>select json_object ( CONCAT('k',t.number%30926%3000 + 0),CONCAT('k',t.number%30926%3000 + 0,t.number%1000000) ) from numbers("number" = "2") t order by 1;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| json_object(concat('k', cast((((number % 30926) % 3000) + 0) as VARCHAR(65533))), concat('k', cast((((number % 30926) % 3000) + 0) as VARCHAR(65533)), cast((number % 1000000) as VARCHAR(65533))), '66') |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"k0":"k00"}                                                                                                                                                                                              |
| {"k1":"k11"}                                                                                                                                                                                              |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.04 sec)

before:

mysql [(none)]>select json_object
    -> (
    -> CONCAT('k',t.number%30926%3000 + 0),CONCAT('k',t.number%30926%3000 + 0,t.number%1000000)
    -> )
    -> from numbers("number" = "2") t;
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.16.10.8)[INTERNAL_ERROR]function json_object can not input null value , JSON documents may not contain NULL member names.
mysql [(none)]>

Release note

None

Check List (For Author)

Check List (For Reviewer who merge this PR)

doris-robot commented 2 days ago

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?
github-actions[bot] commented 2 days ago

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] commented 2 days ago

PR approved by anyone and no changes requested.

zhangstar333 commented 1 day ago

run buildall

doris-robot commented 1 day ago

TeamCity be ut coverage result: Function Coverage: 38.03% (9902/26035) Line Coverage: 29.22% (82841/283514) Region Coverage: 28.35% (42539/150075) Branch Coverage: 24.91% (21565/86588) Coverage Report: http://coverage.selectdb-in.cc/coverage/162b7538e914810a77717a4eabc1b1428990b366_162b7538e914810a77717a4eabc1b1428990b366/report/index.html

github-actions[bot] commented 1 day ago

PR approved by at least one committer and no changes requested.

doris-robot commented 1 day ago
TPC-H: Total hot run time: 40204 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 162b7538e914810a77717a4eabc1b1428990b366, data reload: false ------ Round 1 ---------------------------------- q1 17730 7490 7314 7314 q2 2052 178 183 178 q3 10856 1096 1150 1096 q4 10395 795 720 720 q5 7631 2796 2772 2772 q6 239 149 147 147 q7 999 631 608 608 q8 9264 1909 1971 1909 q9 6619 6486 6449 6449 q10 6972 2316 2338 2316 q11 462 262 258 258 q12 426 222 222 222 q13 17775 3122 3066 3066 q14 248 210 210 210 q15 586 528 535 528 q16 679 585 588 585 q17 1004 590 478 478 q18 7521 6741 6647 6647 q19 1337 1054 1000 1000 q20 495 180 189 180 q21 4064 3213 3334 3213 q22 388 319 308 308 Total cold run time: 107742 ms Total hot run time: 40204 ms ----- Round 2, with runtime_filter_mode=off ----- q1 7263 7245 7288 7245 q2 336 226 228 226 q3 2923 2976 2951 2951 q4 2063 1855 1834 1834 q5 5651 5715 5750 5715 q6 226 139 147 139 q7 2275 1815 1796 1796 q8 3438 3546 3561 3546 q9 8834 8956 8939 8939 q10 3609 3565 3551 3551 q11 639 508 506 506 q12 786 603 635 603 q13 12410 3188 3190 3188 q14 305 267 272 267 q15 583 527 520 520 q16 680 656 656 656 q17 1850 1639 1640 1639 q18 8338 7836 7531 7531 q19 1717 1664 1548 1548 q20 2126 1895 1883 1883 q21 5644 5520 5469 5469 q22 656 581 572 572 Total cold run time: 72352 ms Total hot run time: 60324 ms ```
doris-robot commented 1 day ago
TPC-DS: Total hot run time: 195814 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 162b7538e914810a77717a4eabc1b1428990b366, data reload: false query1 1244 950 914 914 query2 6245 2125 2103 2103 query3 10984 4003 4007 4003 query4 67770 29018 23596 23596 query5 4937 460 473 460 query6 415 181 204 181 query7 5537 305 295 295 query8 318 232 232 232 query9 8812 2705 2684 2684 query10 457 256 258 256 query11 17208 15286 15940 15286 query12 152 102 108 102 query13 1486 425 435 425 query14 10542 7724 6885 6885 query15 215 196 183 183 query16 7103 481 459 459 query17 997 554 542 542 query18 1736 291 282 282 query19 190 142 147 142 query20 114 113 107 107 query21 197 101 99 99 query22 4582 4385 4462 4385 query23 34897 34211 34087 34087 query24 5425 2394 2456 2394 query25 493 409 403 403 query26 648 183 156 156 query27 1657 281 287 281 query28 4226 2439 2409 2409 query29 650 406 409 406 query30 208 144 142 142 query31 1016 813 811 811 query32 66 57 57 57 query33 385 283 285 283 query34 898 526 513 513 query35 833 752 729 729 query36 1090 966 957 957 query37 122 76 74 74 query38 4390 4270 4267 4267 query39 1454 1436 1458 1436 query40 193 99 110 99 query41 45 48 47 47 query42 101 102 100 100 query43 528 490 475 475 query44 1185 824 822 822 query45 178 170 163 163 query46 1121 690 714 690 query47 1937 1857 1891 1857 query48 433 307 318 307 query49 714 394 392 392 query50 829 383 409 383 query51 7244 7153 7193 7153 query52 99 89 88 88 query53 264 176 179 176 query54 544 403 403 403 query55 78 76 85 76 query56 249 237 251 237 query57 1298 1156 1122 1122 query58 224 215 217 215 query59 3183 2989 2925 2925 query60 262 248 259 248 query61 115 111 108 108 query62 778 685 669 669 query63 219 193 186 186 query64 1446 667 632 632 query65 3286 3204 3231 3204 query66 699 308 335 308 query67 15878 15520 15621 15520 query68 2882 572 578 572 query69 412 260 260 260 query70 1209 1142 1158 1142 query71 366 245 257 245 query72 5990 4125 4139 4125 query73 785 374 386 374 query74 9618 8980 9339 8980 query75 3408 2705 2773 2705 query76 1492 1124 971 971 query77 466 276 275 275 query78 10550 9384 9392 9384 query79 2394 609 609 609 query80 1391 432 447 432 query81 528 240 249 240 query82 1334 120 122 120 query83 186 151 153 151 query84 275 71 72 71 query85 990 299 297 297 query86 428 298 282 282 query87 4749 4651 4585 4585 query88 3845 2263 2220 2220 query89 419 292 293 292 query90 1972 186 190 186 query91 133 103 105 103 query92 64 48 49 48 query93 2797 530 539 530 query94 883 296 296 296 query95 358 251 253 251 query96 647 272 284 272 query97 2930 2739 2659 2659 query98 211 199 205 199 query99 1859 1319 1307 1307 Total cold run time: 318400 ms Total hot run time: 195814 ms ```
doris-robot commented 1 day ago
ClickBench: Total hot run time: 32.82 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 162b7538e914810a77717a4eabc1b1428990b366, data reload: false query1 0.03 0.03 0.02 query2 0.07 0.03 0.04 query3 0.23 0.08 0.06 query4 1.61 0.10 0.11 query5 0.45 0.38 0.41 query6 1.16 0.66 0.66 query7 0.01 0.02 0.02 query8 0.04 0.04 0.03 query9 0.57 0.52 0.49 query10 0.53 0.53 0.56 query11 0.15 0.11 0.11 query12 0.14 0.11 0.11 query13 0.62 0.60 0.60 query14 2.86 2.76 2.84 query15 0.90 0.84 0.84 query16 0.38 0.36 0.40 query17 1.06 1.07 0.98 query18 0.20 0.20 0.20 query19 1.91 1.86 1.99 query20 0.01 0.01 0.02 query21 15.39 0.58 0.58 query22 3.07 2.98 2.05 query23 17.01 0.95 0.89 query24 3.54 1.79 0.96 query25 0.17 0.14 0.08 query26 0.54 0.13 0.15 query27 0.05 0.04 0.04 query28 9.90 1.09 1.07 query29 12.56 3.24 3.22 query30 0.24 0.06 0.07 query31 2.86 0.40 0.38 query32 3.27 0.47 0.47 query33 2.97 3.08 3.00 query34 17.02 4.48 4.52 query35 4.59 4.48 4.58 query36 0.68 0.50 0.48 query37 0.09 0.07 0.06 query38 0.05 0.03 0.04 query39 0.03 0.02 0.02 query40 0.16 0.13 0.12 query41 0.07 0.02 0.02 query42 0.03 0.02 0.02 query43 0.04 0.03 0.03 Total cold run time: 107.26 s Total hot run time: 32.82 s ```