search

apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine
https://datafusion.apache.org/ballista
Apache License 2.0
1.57k stars 198 forks source link

Refactor the TaskDefinition by changing encoding execution plan to the decoded one #817

Closed yahoNanJing closed 1 year ago

yahoNanJing commented 1 year ago

Which issue does this PR close?

Closes #793.

Rationale for this change

I think it's better for the plan decoding process inside the grpc interface so that we can detect failure as quickly as possible. And it's kind of meaningless to send a task with an illegal plan to the channel for future processing.

What changes are included in this PR?

  1. The patch of #743 is reverted
  2. The encoded execution plan in the TaskDefinition is changed to the decoded one.
  3. The error handling of run_task in the executor_server is refined

Are there any user-facing changes?

yahoNanJing commented 1 year ago

Hi @thinkharderdev and @Dandandan, could you help review this PR?