search

camsas / firmament

The Firmament cluster scheduling platform
Apache License 2.0
412 stars 77 forks source link

Whare-Map cost model implementation #31

Closed ms705 closed 9 years ago

ms705 commented 9 years ago

This PR brings the complete implementation of the Whare-Map cost model into the master branch.

This cost model is based on an ISCA 2013 paper and uses instructions-per-second (IPS) as an indirect metric for both machine type affinity and co-location interference. The cost model transforms the Whare-Map approach into a flow scheduling cost model, and as a side effect improves its effectiveness: the original approach used a stochastic hill climbing approach and was not guaranteed to pick the best option, while the flow scheduler cost model always does.

The implementation supports both the Whare-M (co-location-agnostic) and Whare-MCs (co-location-aware) approaches; Whare-MCs is enabled and controlled using the --num_pref_arcs_agg_to_res command line flag.

In experiments, this cost model improves the runtime of synthetic test workloads by 2-4x in the median as it avoids interference and matches tasks to the most appropriate machine.

ms705 commented 9 years ago

Code reviews for the commits in this PR have all been approved.