search

open-telemetry / opentelemetry-helm-charts

OpenTelemetry Helm Charts
https://opentelemetry.io
Apache License 2.0
391 stars 469 forks source link

How to figure out or calculate the Resource limits to be set for opentelemetry-collector #872

Open shreyas-h-d opened 1 year ago

shreyas-h-d commented 1 year ago

Hi,

I'm using opentelemtry-collector via deployment mode in my kubernetes cluster. I'm trying to figure out what is the least amount of resource limits I can set to this opentelemetry-collector so that it does not get OOM killed. Can someone please help me understand how I can figure out or calculate the resource limits I need to set.

I'm using prometheus in the receiver part of the opentelemetry-collector's pipeline and trying to scrape apiserver_request_total metric from the kube-apiserver and couple of metrics from kube-state-metrics. Later perform some operations using metricstransform in the processor part of the otel pipeline.

Thanks!

maczg commented 1 year ago

Just my two cents. Setting limits on any workload is a hot topic that's widely discussed for many reasons. Especially when it comes to memory, which is a non-compressible resource and therefore can't be stretched the way CPU can be (by the way, CPU throttling is a silent enemy).

What values have you set? In my case, using a Helm chart for deployment, i have usage per pod like this

# kubectl top pod otel-demo-otelcol-$ID
NAME                                 CPU(cores)   MEMORY(bytes)   
otel-demo-otelcol-84f4467bf6-5rjd5   19m          74Mi

As for memory limits, I've set it to 125Mi and haven't encountered any issues so far.

TylerHelmuth commented 1 year ago

This doc may also be helpful: https://opentelemetry.io/docs/collector/scaling/