search

aws-samples / amazon-cloudwatch-container-insights

CloudWatch Agent Dockerfile and K8s YAML templates for CloudWatch Container Insights.
MIT No Attribution
163 stars 107 forks source link

Too high cpu request for fluentbit daemonset #118

Open mukundjalan opened 1 year ago

mukundjalan commented 1 year ago

As per the example provided on https://github.com/aws-samples/amazon-cloudwatch-container-insights/blob/master/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/fluent-bit/fluent-bit.yaml, fluentbit is requesting 500m of CPU inside EKS.

    resources:
        limits:
          memory: 200Mi
        requests:
          cpu: 500m
          memory: 100Mi

According to fluentbit documentation and statistics provided by AWS, fluentbit is much less CPU & memory intensive as compared to fluentd, so why is this example using a higher CPU than the fluentd counterparts from the past? I would like to know how do I decide how much CPU/memory must be allocated to the daemonset pod because with the above configuration, it is almost impossible for me to use a t3.medium or t3.large instance as most of its CPU is consumed by the daemonset pod leaving none to little room for real workload pods.

aguzmans commented 8 months ago

I have a similar issue. I have a tiny cluster used for very light work, it normally runs about 2 to 5 nodes with autoscaler and the fluent-bit consumes more CPU than any other workload in the cluster. when the cluster is on 2 nodes with not a lot of load it consumes about 1vCPU of the EC2 instances. Not sure why or how to determine why!?