Support max-allowed Pods for nodes

[islinwb]

xref:

• https://github.com/kubernetes-sigs/poseidon/issues/88
• https://github.com/kubernetes-sigs/poseidon/pull/112

[m1093782566]

/cc @shivramsrivastava

[islinwb]

@shivramsrivastava The PR's ready.

[islinwb]

@shivramsrivastava PTAL

[nrshrivatsan]

TL;DR

Schedulers should be cognizant about

1. Workload-Priority
2. Predicates/Prerequisite for Workloads

Priority

Instead of a discrete logic of MAX permissible Pods on a Node, would you encourage us having a more condition tree based scheduling?

Node ---- HAS ----> Pods ---> have ---> Priority

Schedulers could honor Pod Priority [ P1, P2, P3 ] along-side maximum permissible number of Pods.

Scheduler ---> Node A ---> Spawn max `P1` Pods ----> Spaw max `P2` Pods

Predicate

Context

It's a good start to have a Rate-Limiting function for workload scheduling. However workloads in Production grade systems tend to have criteria like

• Affinity/Anti-affinity
• Taint tolerant workloads
• Volume dependent
• Fault-domain resilient

Suggestion

If we abstract the Rate-limit to a be a type of Predicate for scheduling, we could have an elegant predicate-based posidon scheduling.

Reasons

• From Business availability stand point, there is a priority for any workload
• MUST have Pods need to be scheduled first, followed by Nice-to-Have pods

@shivramsrivastava @m1093782566 @islinwb - RFC

[deepak-vij]

Thanks for your feedback. This is exactly we currently support as part of Poseidon/Firmament scheduler. Node Filtering (Hard Constraints) -> Priority/Scoring (Soft Constraints) -> Taints -> Volume dependency. We are about to incorporate max. pods/node capability to go along with all of the above. If this is what you meant.

[nrshrivatsan]

@deepak-vij thanks for the comment. Could I request for a few reference hyperlinks to the documentation for Hard & Soft constraints?

[shivramsrivastava]

@nrshrivatsan

1) Taints and Tolerations 2) Pod Level Affinity/Anti-Affinity 3) Node Level Affinity/Anti-Affinity

4) Volumes dependent workloads: We are currently working on this one, we support only few volumes types and only support pre-bound PVC's and do not support dynamic binding/storage classes. We will publish a document on this soon.

Can you please elaborate on the Rate-Limiting predicate? Are you suggesting to introduce Rate-Limiting for pods scheduling, as a part of predicate operation?

[nrshrivatsan]

@shivramsrivastava Love the links!

Rate Limiting

• Set of Pods could satisfy Predicates
• While scheduling these Pods, since resource quotas might be confined, it would be wise if the Scheduler schedules them in a rate-limited fashion
• Value gains of Rate-limiting : determinism in pod-scheduling rate.