search

Graylog2 / collector-sidecar

Manage log collectors through Graylog
https://www.graylog.org/
Other
268 stars 56 forks source link

Is there a limit to the number of log files for collector-sidecar tail? #386

Closed WangHouLing closed 4 years ago

WangHouLing commented 4 years ago

Problem description

Hi all, Excuse me. I am using collector-sidecar to collect log data(Filebeat backend), there will be more log files to collect.So I need to know if the collector-sidecar has a limit on the number of log files(The "paths" configuration item in filebeat.yml). How many sub-directory files can be collected in a parent directory? What is the performance of collector-sidecar when there are a large number of files in the subdirectory? I would very appreciate it if you can help me.

Steps to reproduce the problem

graylog server web: image filebeat.yml: image

Environment

mariussturm commented 4 years ago

Hi, there is no technical limitation build-in. The limit is more defined by the server capacity. Filebeat will need more CPU and Memory resources when it collects more files. But that's nothing we can affect.