coderabbitai[bot]
commented
1 week ago Walkthrough
The recent update introduces substantial asynchronous capabilities into the project by changing multiple synchronous functions to asynchronous, enabled by the async-recursion dependency in Cargo.toml. It also simplifies the locking mechanisms in worker.rs and removes serialization traits from various structs, streamlining data handling and enhancing concurrency control.
Changes
| File | Change Summary |
|---|---|
src/flow/Cargo.toml |
Added async-recursion = "1.0" dependency. |
src/flow/src/adapter/worker.rs |
Refactored imports, simplified locks, refactored async mechanisms. |
src/flow/src/expr/scalar.rs |
Made try_from_raw_fn asynchronous with awaited get_fn_impl(). |
src/flow/src/transform.rs |
Converted sql_to_flow_plan to asynchronous. |
src/flow/src/transform/aggr.rs |
Changed various TypedExpr and AggregateExpr functions to asynchronous. |
src/flow/src/transform/expr.rs |
Converted multiple functions to asynchronous. |
src/flow/src/transform/literal.rs |
Added .await to TypedPlan::from_substrait_plan in tests. |
src/flow/src/transform/plan.rs |
Changed several functions to asynchronous, including recursive functions. |
src/flow/src/expr/func.rs |
Removed Serialize and Deserialize from UnmaterializableFunc enum. |
src/flow/src/expr/linear.rs |
Removed serialization traits from MapFilterProject and SafeMfpPlan structs. |
src/flow/src/expr/relation.rs |
Removed serialization traits from AggregateExpr. Added serde default for distinct. |
src/flow/src/plan.rs |
Removed serialization traits from TypedPlan and Plan structs. |
src/flow/src/plan/join.rs |
Removed serialization traits from multiple Join related structs. |
src/flow/src/plan/reduce.rs |
Removed serialization traits from multiple Reduce related structures. |
src/flow/src/utils.rs |
Removed serialization traits from KeyExpiryManager and Arrangement structs. |
Sequence Diagram(s)
sequenceDiagram
participant Client
participant Worker
participant InterThreadCallClient
participant InterThreadCallServer
participant RequestHandler
Client->>+Worker: Send Request
Worker->>+InterThreadCallClient: Forward Request
InterThreadCallClient->>+RequestHandler: Handle Request asynchronously
RequestHandler-->>-InterThreadCallClient: Send Response
InterThreadCallClient-->>-Worker: Receive Response
Worker-->>-Client: Send ResponsePoem
In fields of code where bytes do play,
Async flows bring brighter day.
Locks removed, the worker sighs,
As tasks now dance beneath the skies.
No more serialize, we cheer and beam,
A simpler, swifter, coding dream. 🌟
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
codecov[bot]
discord9
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
make
from_substrait_*async&refactor flow worker handle implPlease explain IN DETAIL what the changes are in this PR and why they are needed:
from_substrait_*asyncblock_onin deser to call async methodoneshotto send back response to requestChecklist
Summary by CodeRabbit
New Features
async-recursiondependency for enhanced async functionality.Refactor
Style
Chores
SerializeandDeserializetraits from multiple structs and enums, indicating a change in serialization strategy.