apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine

https://datafusion.apache.org/ballista

Apache License 2.0

1.53k stars 194 forks source link

Add support for delegating to other query engines #685

Closed andygrove closed 1 year ago

andygrove commented 1 year ago

Which issue does this PR close?

Closes https://github.com/apache/arrow-ballista/issues/684

Rationale for this change

Add support for using Ballista as a scheduler for execution engines other than DataFusion

What changes are included in this PR?

Add ability to plug in execution engine in executor

There is a corresponding PR in PyBallista that shows how this is used to support Python execution: https://github.com/apache/arrow-ballista-python/pull/19

Are there any user-facing changes?