search

splunk / docker-logging-plugin

Splunk Connect for Docker is a Docker logging plugin that allows docker containers to send their logs directly to Splunk Enterprise or a Splunk Cloud deployment.
Apache License 2.0
65 stars 25 forks source link

Fix high memory cpu #34

Closed sharonx closed 6 years ago

dtregonning commented 6 years ago

@sharonx Changes look good.

If extended testing results show reduced cpu/mem profiles - merge.

sharonx commented 6 years ago

Yes idle cpu/memory after containers exit went down as compared to the data before. Starting from 2:30am in the chart ran the test with this fix. Ran it with forced GC, memory went down even further (~2.5% memory usage)

image

tonytamsf commented 6 years ago

So the memory went down by ~50% ?

sharonx commented 6 years ago

@tonytamsf memory went down by ~50% in the graph but could go down even more with forced GC (OS doesn't GC everything until needed).

The root cause of the cpu and memory issue is that we didn't return from the go routines once a container exits (each container launches 2 routines). The buffer used in the go routines can't be garbage collected either.