RHAM worker nodes memory overcomit

stormshift / support

This repo should serve as a central source for reporting issues with stormshift

GNU General Public License v3.0

3 stars 0 forks source link

RHAM worker nodes memory overcomit #59

Closed DanielFroehlich closed 2 years ago

DanielFroehlich commented 2 years ago

RHACM worker nodes have 16 GB RAM and are heavily memory overcommited:

> oc describe node/rhacm-ncrfd-worker-0-qqcnr
  memory             14849Mi (99%)  23128Mi (155%)

That leads to issues during cluster upgrade (hang during reboots, nodes not ready). Solution: Bump from 16GB to 48GB memory

github-actions[bot] commented 2 years ago

Heads up @cluster/rhacm-admin - the "cluster/rhacm" label was applied to this issue.

rbo commented 2 years ago

Looks good

oc get nodes -l node-role.kubernetes.io/worker -o name | xargs -I % -n1 oc describe % | grep '^  memory'
  memory:             49426308Ki
  memory:             48275332Ki
  memory             11841Mi (25%)  34752Mi (73%)
  memory:             49426308Ki
  memory:             48275332Ki
  memory             12461Mi (26%)  12092Mi (25%)
  memory:             49426308Ki
  memory:             48275332Ki
  memory             12110Mi (25%)  9700Mi (20%)

DanielFroehlich commented 2 years ago

actually, 32GB will be enough. scaling down a bit....

DanielFroehlich commented 2 years ago

... done, we go with 32GB until this is not enough. Closing...