apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine
https://datafusion.apache.org/ballista
Apache License 2.0
1.49k stars 190 forks source link

The decoded execution plan at the executor side cannot be directly shared. #793

Closed yahoNanJing closed 1 year ago

yahoNanJing commented 1 year ago

Describe the bug

It's caused by the #743. Only unchanged things can be shared. However, the execution plan is not. Its metrics will be changed during task execution. Since the decoded execution plan is directly shared by multiple tasks, the plan metrics is also shared across multiple tasks, which is not correct.

To Reproduce

Expected behavior

Additional context