search

apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine
https://datafusion.apache.org/ballista
Apache License 2.0
1.49k stars 190 forks source link

Design: Implement bubble execution mode. #408

Open liurenjie1024 opened 1 year ago

liurenjie1024 commented 1 year ago

Is your feature request related to a problem or challenge? Please describe what you are trying to do. Currently ballista uses schedules stage by stage, which is great for fault tolerance, but inefficient for adhoc olap execution. Bubble execution mode is to make ballista more efficient for adhoc olap queries which doesn't require fault tolerance.

Describe the solution you'd like The detailed design can be found here, welcome to comment and share your thoughts.

Describe alternatives you've considered No.

Additional context No.

r4ntix commented 1 year ago

Very good proposal. Can you add some designs of popular systems such as Presto and Clickhouse for comparison?

liurenjie1024 commented 1 year ago

Very good proposal. Can you add some designs of popular systems such as Presto and Clickhouse for comparison?

Good suggestion, I'll add later.