Open StepanKuksenko opened 4 years ago
So something doesn't seem right.
@johnSchnake
still have this problem in all our clusters in GKE. When i limit memory usage for eventrouter pod memory usage increases during a time, when memory rests to limit pod restarts with "OOMKilled" reason. There are 247 restarts during seven days.
State: Running
Started: Thu, 23 Jan 2020 13:00:54 +0300
Last State: Terminated
Reason: OOMKilled
Exit Code: 137
Started: Thu, 23 Jan 2020 12:48:53 +0300
Finished: Thu, 23 Jan 2020 13:00:51 +0300
Ready: True
Restart Count: 247
Limits:
cpu: 100m
memory: 256Mi
Requests:
cpu: 100m
memory: 256Mi
Same problem here...
involved_object_name is a label with high cardinality
kubernetesWarningEventCounterVec = prometheus.NewCounterVec(prometheus.CounterOpts{
Name: "heptio_eventrouter_warnings_total",
Help: "Total number of warning events in the kubernetes cluster",
}, []string{
"involved_object_kind",
"involved_object_name",
"involved_object_namespace",
"reason",
"source",
})
So something doesn't seem right. @johnSchnake
Eventrouter is constantly increasing the amount of memory consumed. In one of our cluster after one month eventrouter uses near 8Gb of memory.
It is the output of top command in eventrouter pod (containter)
I will configure limits. But what will happen if eventrouter reaches the memory limit? Will it work correctly. And why does it use so much memory?