apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine
https://datafusion.apache.org/ballista
Apache License 2.0
1.57k stars 198 forks source link

Support for speculative execution tasks #863

Open smallzhongfeng opened 1 year ago

smallzhongfeng commented 1 year ago

Is your feature request related to a problem or challenge? Please describe what you are trying to do. At present, we have encountered the case of slow nodes when some nodes write to hdfs when ballista is executing tasks, so we plan to support speculative execution similar to spark. Then I would like to discuss the feasibility of this plan with you.