Open NamanChandra1 opened 1 month ago
Describe the bug A clear and concise description of what the bug is.
To Reproduce Steps to reproduce the behavior:
Configuration:
logstash.conf: |- input { kafka { consumer_threads => 5 bootstrap_servers => "kafka:9092" decorate_events => basic group_id => "logstash" topics => ["kube-logs"] codec => "json" } http { port => 8000 type => "healthcheck" } } filter { mutate { remove_field => ["[log][file][path]", "[log][offset]"] } if [kubernetes][labels][role] { mutate { add_field => { "kubernetes.annotations.role" => "%{[kubernetes][labels][role]}" } } } mutate { add_field => { "partition" => "%{[@metadata][kafka][partition]}" } remove_field => [ "host","event" ] add_field => { "index_label" => "%{[kubernetes][container][name]}"} add_field => { "kubernetes.pod_name" => "%{[kubernetes][pod][name]}"} add_field => { "kubernetes.container_name" => "%{[kubernetes][container][name]}"} add_field => { "kubernetes.pod_id" => "%{[kubernetes][pod][uid]}"} add_field => { "kubernetes.docker_id" => "%{[container][id]}" } add_field => { "kubernetes.namespace_name" => "%{[kubernetes][namespace]}"} add_field => { "kubernetes.container_image" => "%{[container][image][name]}"} add_field => { "kubernetes.host" => "%{[kubernetes][node][name]}"} } mutate { remove_field => ["[kubernetes][node][labels]", "[kubernetes][pod][uid]","[kubernetes][pod][name]","[agent]","[kubernetes][namespace_uid]", "[container][id]", "[kubernetes][replicaset]","[container][image][name]", "[kubernetes][namespace]" , "[kubernetes][namespace_labels]", "[kubernetes][container][name]", "[kubernetes][node]"] } } output { if "_jsonparsefailure" not in [tags] { opensearch { hosts => ["https://admin:admin@opensearch-cluster-master.logging.svc.cluster.local:9200"] index => "%{index_label}-new-%{+YYYY.MM.dd}" user => "admin" password => "admin" ssl => true ssl_certificate_verification => false } } } logstash.yml: | http.host: 0.0.0.0 pipeline.ecs_compatibility: disabled log.level: debug
Expected behavior Push logs successfully to Opensearch with memory cleanup.
Screenshots If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context Add any other context about the problem here.
May not be actually a leak, these processes may try to use all the memory they have. Does it eventually run out of memory in your case?
Describe the bug A clear and concise description of what the bug is.
To Reproduce Steps to reproduce the behavior:
Configuration:
Expected behavior Push logs successfully to Opensearch with memory cleanup.
Screenshots If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context Add any other context about the problem here.