BCDevOps / developer-experience

This repository is used to track all work for the BCGov Platform Services Team (This includes work for: 1. Platform Experience, 2. Developer Experience 3. Platform Operations/OCP 3)
Apache License 2.0
8 stars 17 forks source link

Research adding limit to ephemeral storage #4645

Open StevenBarre opened 9 months ago

StevenBarre commented 9 months ago

Describe the issue https://kubernetes.io/blog/2022/09/19/local-storage-capacity-isolation-ga/ Look into how to prevent emptyDir from going wild

What is the Value/Impact? Improved cluster stability

What is the plan? How will this get completed? Review docs Make a plan to do it in prod

Identify any dependencies Platform Services PO

Definition of done Plan on how to move forward

vivekratan88 commented 2 months ago

some extra notes:

https://docs.openshift.com/container-platform/4.15/storage/understanding-ephemeral-storage.html#storage-ephemeral-storage-manage_understanding-ephemeral-storage

vivekratan88 commented 1 month ago

https://kubernetes.io/docs/concepts/storage/ephemeral-volumes/

vivekratan88 commented 1 month ago

confirmed this is to be tested on entire namespace level so something like this should be used. apiVersion: v1 kind: LimitRange metadata: name: storage-limit-range spec: limits:

Will get accurate one made for my testing.

vivekratan88 commented 2 weeks ago

fixed it seems like container was closing quickly, hence added a bit of delay seems to have fixed the issue for now. apiVersion: v1 kind: Pod metadata: name: frontend namespace: be1c6b-test spec: containers:

vivekratan88 commented 1 week ago

busybox fill container is crashing looping looking into that.