Improve ballista performance

[yahoNanJing]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

To be an efficient distributed execution platform, there're still many aspects we need to improve.

Describe the solution you'd like

• For scheduler, efficient task scheduling
  • [x] #133 #134
  • [x] #131 #135
  • [x] #137 #147
  • [x] #144 #145
  • [x] #138 #141
  • [x] #149 #150
  • [x] #130 #153
  • [x] #179 #181
  • [x] #136 #156
  • [x] #208 #256
  • [x] #209 #255
  • [x] #391 #392
  • [x] #142 #393
  • [x] #389 #394
  • [x] #371 #395
  • [x] #132 #396
  • [ ] #397 #398
  • [x] https://github.com/apache/arrow-ballista/issues/243

Describe alternatives you've considered

Additional context

[andygrove]

I added https://github.com/apache/arrow-ballista/issues/243

[yahoNanJing]

With all the PRs merged, we can achieve a relatively high throughput in RoundRobinLocal SlotsPolicy. For a 200ms query with 200 concurrency running 5 minutes, the average query latency is also around 200ms, which proves the task scheduling is very efficient and should not be a bottleneck in a high throughput system.

And for a large query with with more than 2000 tasks in the first stage, the query latency is improved from more than 10s to around 2.5s.

Hi @thinkharderdev and @andygrove, if you have time, you can also help verify on it.

[yahoNanJing]

Since Dan may not have time for reviewing on these PRs, Andy could you at other guys who may be interested on these PRs? Thanks very much.