Add a scheduler to create nodes based on pod requests

AkihiroSuda / instance-per-pod

Create a dedicated IaaS instance per Pod to mitigate container breakout (including CPU vulnerabilities depending on the instance type)

Apache License 2.0

22 stars 2 forks source link

Open craigbox opened 4 years ago

craigbox commented 4 years ago

Consider this:

webhook sets custom scheduler
scheduler looks at pod requests/limits for new pods
(optional) scheduler looks at pods that will run as DaemonSets
scheduler creates & adds a custom sized node for that pod (optionally: plus its DaemonSets)
scheduler attaches pod to that node

You now have both "one instance per pod" for hypervisor isolation, and "only paying for resource you use" due to the custom pod/VM shapes.

44past4 commented 4 years ago

Newly created node should be terminated as soon as the pod for which it has been created is terminated.

AkihiroSuda commented 4 years ago

Thanks for the suggestion, I also considered that, but implementing IaaS drivers for several cloud providers might be hard.

"only paying for resource you use" due to the custom pod/VM shapes.

Maybe this can be achieved with the current Cluster Autoscaler model by using multiple node pools.

AkihiroSuda commented 4 years ago

Also, AFAIK, GKE doesn't support adding custom GCE nodes to the cluster?

craigbox commented 4 years ago

Understood, and while it might change the shape of your project to do so, I think it would be an interesting thought experiment.

You would have to create a node pool for each size of node. I checked with PM and we can scale up to 100 node pools at this point in time.