StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
This pr introduce some optimization for looking up dictionary:
reserve the memory for the dest column which using for saving the result
to avoid the memory reallocation.
Using _append_value instead of creating Datum for appending.
If the value type is Slice, save the slice first and append when to loop
is finished. Avoid polluting the cpu cache for the prefetched content.
In our test case, for 1BE(16cpu, 128GB) + 1FE, here is the expression computation time:
key: int -> value: (int) : 572ms(before optimization) vs. 268ms(after optimization)
key: int -> value: (int, int) : 732ms(before optimization) vs. 395ms(after optimization)
key: int -> value: (int, int, varchar(length 128)) : 1s211ms(before optimization) vs. 744ms(after optimization)
Fixes #issue
What type of PR is this:
[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool
Does this PR entail a change in behavior?
[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
[x] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)
[ ] I have added documentation for my new feature or new function
[x] This is a backport pr
Bugfix cherry-pick branch check:
[x] I have checked the version labels which the pr will be auto-backported to the target branch
[x] 3.3
[ ] 3.2
[ ] 3.1
[ ] 3.0
[ ] 2.5
This is an automatic backport of pull request #47050 done by Mergify.
[Enhancement] optimize lookup for dictionary
This pr introduce some optimization for looking up dictionary:
reserve the memory for the dest column which using for saving the result
to avoid the memory reallocation.
Using _append_value instead of creating Datum for appending.
If the value type is Slice, save the slice first and append when to loop
is finished. Avoid polluting the cpu cache for the prefetched content.
In our test case, for 1BE(16cpu, 128GB) + 1FE, here is the expression computation time:
key: int -> value: (int) : 572ms(before optimization) vs. 268ms(after optimization)
key: int -> value: (int, int) : 732ms(before optimization) vs. 395ms(after optimization)
key: int -> value: (int, int, varchar(length 128)) : 1s211ms(before optimization) vs. 744ms(after optimization)
Fixes #issue
What type of PR is this:
[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool
Does this PR entail a change in behavior?
[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
[x] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)
[ ] I have added documentation for my new feature or new function
[Enhancement] optimize lookup for dictionary
This pr introduce some optimization for looking up dictionary:
In our test case, for 1BE(16cpu, 128GB) + 1FE, here is the expression computation time: key: int -> value: (int) : 572ms(before optimization) vs. 268ms(after optimization) key: int -> value: (int, int) : 732ms(before optimization) vs. 395ms(after optimization) key: int -> value: (int, int, varchar(length 128)) : 1s211ms(before optimization) vs. 744ms(after optimization)
Fixes #issue
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check:
This is an automatic backport of pull request #47050 done by Mergify. [Enhancement] optimize lookup for dictionary
This pr introduce some optimization for looking up dictionary:
In our test case, for 1BE(16cpu, 128GB) + 1FE, here is the expression computation time: key: int -> value: (int) : 572ms(before optimization) vs. 268ms(after optimization) key: int -> value: (int, int) : 732ms(before optimization) vs. 395ms(after optimization) key: int -> value: (int, int, varchar(length 128)) : 1s211ms(before optimization) vs. 744ms(after optimization)
Fixes #issue
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist: