Optimize RAM by not storing all pod metadata in the internal maps.

twosigma / Cook

Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark

Apache License 2.0

337 stars 63 forks source link

Optimize RAM by not storing all pod metadata in the internal maps. #2168

Closed scrosby closed 2 years ago

scrosby commented 2 years ago

Changes proposed in this PR

Do not save all of the pod data when we're tracking kubernetes state.

Why are we making these changes?

If you don't save it, it doesn't clutter up RAM, and in particular, clutter up oldgen. The worst case for GC is objects that live a long time but eventually die. This hopefully cuts the rate of those objects, at least in the case of Pod data.

scrosby commented 2 years ago

Doesn't save anything (we store another copy, and have to store it), and we use the spec in other spots in the code. Do not merge.