Heterogeneity and Dynamicity of Clouds at Scale: Google Trace Analysis

gaocegege commented 7 years ago

at15 commented 7 years ago

好像这个 trace 是可以下载的.... 我估计毕设可能跟 tracing 也有点关系

gaocegege commented 7 years ago

嗯是的，google trace

gaocegege commented 7 years ago

这篇文章对做调度器的人而言指导意义还是很大的, 建议全文阅读.

其中比较有意义的结论有:

任务的 duration 和 usage 很难用统计学分布的方式表示出来, 写调度的 evaluation 最好是直接 sample, 如果不行:

power-law-like heavy-tailed distribution for task usage and task durations with similar distributions heavily-biased towards powers-of-two and ten for user-set job parameters like task counts and resource requests may approximate the observed behavior.
集群的变化是非常频繁的, 40% 的机器都 down 过至少一次, rescheduling 和 new task scheduling 都是很多的, 最多的时候一秒调度几百个任务, 所以调度器的延迟真的可以忽略么? 未必可以
小任务是最多的:

75% of jobs consist of only one task, half of the jobs run for less than 3 minutes, and most jobs request less than 5% of the average machine’s resources
预测集群利用率的时候, 排除运行时间低于 2 小时的影响不大
虽然 long-running 的 job 不多, 但是基本每个用户都会跑至少一个(615 of the 925 users)
The Google trace provides hard constraints for approximately 6% of all tasks submitted to the scheduler.

dyweb / papers-notebook