In an effort to provide cost-optimization on the cluster scaling, I've been trying to schedule worker pods on to different sized nodes. I have two nodepools: small and large. I set the default pod limits to 1vcpu and 11GB RAM (wrt RAM, two fit on a small node and four would fit on to a large node), however, I also want some users to, through the cluster.options configuration when starting a cluster, to request higher limits (3vcpus and 21GB RAM for a full small node or even 7vcpu and 45GB RAM for a large node).
Below is the NodeAffinity solution that kind of works, however, when a large enough cluster is scaled up with the default limits (1vcpu and 11GB RAM), sometimes this triggers the scale up of the larger nodes. This isn't a surprise due to the kube-scheduler and the non-guarantee of the preferredDuringSchedulingIgnoredDuringExecution preferences.
So, I was wondering, is it feasible to have multiple profiles for worker pods? Something that would effectively extend the cluster profile options here.
Otherwise, perhaps someone may have some better insight in how to achieve what I'm trying to accomplish.
In an effort to provide cost-optimization on the cluster scaling, I've been trying to schedule worker pods on to different sized nodes. I have two nodepools: small and large. I set the default pod limits to 1vcpu and 11GB RAM (wrt RAM, two fit on a small node and four would fit on to a large node), however, I also want some users to, through the
cluster.options
configuration when starting a cluster, to request higher limits (3vcpus and 21GB RAM for a full small node or even 7vcpu and 45GB RAM for a large node).Below is the
NodeAffinity
solution that kind of works, however, when a large enough cluster is scaled up with the default limits (1vcpu and 11GB RAM), sometimes this triggers the scale up of the larger nodes. This isn't a surprise due to thekube-scheduler
and the non-guarantee of thepreferredDuringSchedulingIgnoredDuringExecution
preferences.So, I was wondering, is it feasible to have multiple profiles for worker pods? Something that would effectively extend the cluster profile options here.
Otherwise, perhaps someone may have some better insight in how to achieve what I'm trying to accomplish.