search

pusher / k8s-spot-rescheduler

Tries to move K8s Pods from on-demand to spot instances
Apache License 2.0
311 stars 42 forks source link

Gpu #43

Closed bhack closed 6 years ago

bhack commented 6 years ago

How this rescheduler will handle GPU?

wonderhoss commented 6 years ago

Could you give a little more background on your question?

If you are referring to EC2 instances with GPU capabilities, then the answer is: it does not.

If some of your nodes have GPU capability and some do not, you need to ensure that you only use the rescheduler with node groups that both have that capacility.

In general, you need to make sure that any workloads schedulable on the "on-demand" node group can also be scheduled on the "spot" node group in order to ensure that no workload ends up unschedulable if evicted.

bhack commented 6 years ago

Yes I meant EC2 instances with GPU. My case is that we have some nodes with GPU and some nodes without.

wonderhoss commented 6 years ago

You define the node group to consider for draining with --on-demand-node-label and the group to shift workload to with --spot-node-label. You can either use the rescheduler only for node groups without GPU or for for groups with.

Hope that helps.

bhack commented 6 years ago

ok thanks