search

park-project / park

MIT License
218 stars 55 forks source link

load_balance: Why are job sizes are being selected according to Pareto distribution? #16

Closed ratnadeepb closed 4 years ago

ratnadeepb commented 4 years ago

I am working with the load balance environment. And I see that job sizes are being selected from a Pareto distribution (shape = 100, scale = 1.5). I notice that these settings are from the "Variance Reduction for Reinforcement Learning in Input-Driven Environments" paper but I could not find a reference as to why this decision was made. I was wondering if someone can clarify why this choice was made?

hongzimao commented 4 years ago

Thanks for the question. This setting generates a heavy-tailed distribution of the workload. Two major reasons for simulating such workload. 1. It's known that a lot of production traffic/workload/job size/packet delay demonstrate heavy tailed behavior. 2. In this load balancing environment, this setting also generates interesting scenarios that you can easily find better policies that perform better than trivial heuristics (e.g., random heuristic, join the shortest queue heuristic, etc.). Hope these help!

ratnadeepb commented 4 years ago

Yes it definitely helps, thanks for the confirmation.