akash-network / support

Akash Support and Issue Tracking
5 stars 3 forks source link

Investigate possibility of fork bomb on provider clusters #230

Open cloud-j-luna opened 4 weeks ago

cloud-j-luna commented 4 weeks ago

Kubernetes by default is not protected against a "fork bomb". A "fork bomb" would essentially be a Pod spawning many processes to the point other workloads get impacted because the controllers will no longer be able to create Pods and they'll be stalled. It could theoretically crash the node. This happens because pods share the host kernel settings. A badly designed application that doesn't clean spawned child processes can easily cause a node to crash because of this. This issue is to investigate if this is an issue that can happen and how we can limit the impact and protect providers as much as possible.

To Reproduce One way to test this would be to create a simple application that infinitely spawns processes to check if a crash happens.

Additional context To check the maximum number of pids run:

cat /proc/sys/kernel/pid_max

The limit on pids can be set on the kubelet configuration.