search

SumoLogic / sumologic-collector-docker

A Sumo Logic collector for Docker.
Apache License 2.0
69 stars 56 forks source link

Effects of a memory limit on Docker Collection #53

Open aleclarson opened 7 years ago

aleclarson commented 7 years ago

I noticed the SumoLogic container created with sumologic/collector:latest takes up 180 MiB of memory. Is there a recommended way of reducing such usage?

Obviously, I could set a memory limit when starting the container, but I'm not aware of what the effects would be.

Would reducing the docker stats polling interval be enough to reduce memory usage?

Thanks.

maimaisie commented 7 years ago

Hi @aleclarson , how many sources do you have in this collector? The number of sources and ingest volume may affect memory usage. You can definitely try reducing the polling interval for your Docker Stats source. Please also take a look at our help page for collector memory usage: https://help.sumologic.com/Send-Data/Collector-FAQs/Collector-Uses-More-than-the-Allocated-Memory

Thanks, Maisie

aleclarson commented 7 years ago

My 2 sources are docker-stats and docker-logs with 3 running containers (besides the sumologic/collector:latest container). This is using 176.8 MiB of memory.

Is that amount of memory usage normal?

maimaisie commented 7 years ago

Has this number been lower on your collector (with previous collector version or any configuration changes on sources), or have you just noticed the memory usage lately? It is very possible this is the normal usage based on your configuration and ingest volume.

aleclarson commented 7 years ago

I'm a new user of SumoLogic, so I can't comment on previous versions.

But I can tell you docker-stats is set to scan each container once per minute. And right now, docker-logs is getting almost nothing in terms of ingest volume.

What about sumologic-collector-docker consumes ~180 MiB of memory when basically idle?

maimaisie commented 7 years ago

Java-based processes can be memory consuming. The .jar dependencies we ship in the collector is ~50 MiB, and the java runtime itself will either direct consume or reserve (at least) 10s more MB. So a collector with no source would already gets to ~100 MiB. With sources configured, ~180 MiB doesn't look very surprising.