Integrating affinity/anti-affinity aware policy into firmament

[yingfeng]

Kubernetes has already supported inter-pod affinity/anti-affinity within its native scheduler, however, such an important feature could not be found within firmament. Another limitation is the support of task preemption, until now TaskPreemptionCost is still empty.

So when the features of affinity/anti-affinity, as well as task preemption could be supported by firmament? And if possible, could technical report be delivered at first?

[timothysc]

It's my understanding that folks may already be working on some of this.

/cc @quinton-hoole

[ghost]

@timothysc Yes, @deepak-vij can provide details.

[deepak-vij]

Yes we do have a conceptual design for supporting affinity/anti-affinity using Firmament scheduler within Kubernetes. Although, it is quite premature to share these details at this time. We are planning to open source this whole work in the coming months and will start sharing details with the community in due course.

At this time, we are currently stabilizing the integration glue code “Poseidon” and are using very simple cost model that does not depend on real machine utilization, but only on the number of tasks placed on machines. Currently we are using this simple cost model for balancing tasks across all the machines. Our plan is to use granular machine usage metrics in support of more advanced cost model/s. As things progress, we will start working with the Scheduler SIG (David Oppenheimer) w.r.t. open sourcing our work for enabling Firmament Scheduler within Kubernetes environment. Thanks.

[ms705]

@yingfeng Task preemption is supported in the existing code base: if a task's unscheduled cost exceeds the unscheduled cost another (running) task would experience if preempted, Firmament will preempt the second task and schedule the first. The mechanics of this have not been tested extensively in the standalone cluster manager, but should work. In the Kubernetes integration, preemption is not yet supported (though I believe it's planned).

As for affinity/anti-affinity, there is some preliminary work in my PhD dissertation (see, e.g., pages 144-145), and I believe @ICGog is currently extending this.

For technical reports describing the system, see:

1. the OSDI 2016 paper;
2. chapters 6-8 of my PhD thesis (p.133-195).

[deepak-vij]

Hi Malte, good to hear from you. I am sure you are aware that we are collaborating with Ionel on this effort. Thanks.

Deepak

[yingfeng]

@ms705 @deepak-vij Thanks for your explaination, hope these features could be available to be integrated into K8S asap~