Closed waynexia closed 6 days ago
Several modifications were made to the query planning and execution logic, focusing on enhanced partition handling, sorting improvements, and optimization of execution plans by removing unnecessary repartitioning nodes. These changes refine data processing flows, such as introducing target_partition
in MergeScanExec
, altering sorting mechanisms, and updating query plans in various test cases.
File(s) or Group | Summary |
---|---|
merge_scan.rs , planner.rs |
Introduced target_partition to MergeScanExec , updating methods like to_stream and execute to utilize partition info. |
range_manipulate.rs |
Updated properties initialization in RangeManipulateExec to derive from input properties. |
Various test files (.result and .sql ) |
Made various updates including removal of RepartitionExec nodes, adding SortPreservingMergeExec , and adjusting sorting parameters. |
sequenceDiagram
participant QueryPlanner
participant MergeScanExec
participant DistExtensionPlanner
actor User
User->>QueryPlanner: Submit Query
QueryPlanner->>DistExtensionPlanner: Plan Execution
DistExtensionPlanner->>DistExtensionPlanner: Fetch Config
DistExtensionPlanner->>QueryPlanner: Return Plan with target_partition
QueryPlanner->>MergeScanExec: Distribute to target_partition
MergeScanExec->>MergeScanExec: Execute with target_partition
MergeScanExec->>User: Return Query Result
In the land of code where data flows, Partitions dance and merge like prose. Sorting now with grace, so fine, Preservation's touch in every line. No more nodes that don't belong, Efficiency sings a joyful song!
✨🐇🌟
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
Attention: Patch coverage is 63.15789%
with 7 lines
in your changes missing coverage. Please review.
Project coverage is 84.58%. Comparing base (
352cc9d
) to head (f80490d
). Report is 1 commits behind head on main.
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
Use
target_partitions
to guide the parallelism ofMergeScanExec
(This was intended)
Checklist
Summary by CodeRabbit
New Features
Bug Fixes
SortPreservingMergeExec
.Performance Improvements
RepartitionExec
steps in query plans.Testing