search

apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine
https://datafusion.apache.org/ballista
Apache License 2.0
1.55k stars 197 forks source link

Design: Implement bubble execution mode. #408

Open liurenjie1024 opened 2 years ago

liurenjie1024 commented 2 years ago

Is your feature request related to a problem or challenge? Please describe what you are trying to do. Currently ballista uses schedules stage by stage, which is great for fault tolerance, but inefficient for adhoc olap execution. Bubble execution mode is to make ballista more efficient for adhoc olap queries which doesn't require fault tolerance.

Describe the solution you'd like The detailed design can be found here, welcome to comment and share your thoughts.

Describe alternatives you've considered No.

Additional context No.

r4ntix commented 2 years ago

Very good proposal. Can you add some designs of popular systems such as Presto and Clickhouse for comparison?

liurenjie1024 commented 2 years ago

Very good proposal. Can you add some designs of popular systems such as Presto and Clickhouse for comparison?

Good suggestion, I'll add later.