search

grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
24.04k stars 3.47k forks source link

The maximum of the log files that promtail can tailing #15115

Open webz8701 opened 4 days ago

webz8701 commented 4 days ago

Hi,

I am monitoring my logs by promtail + loki.

In my system, there are too many log files (more than 10,000, and continual increasing). The log files organized is similar with /var/log/application/{year}/{date}/{task_id}.log, so the log file numbers increased day by day. When i applied promtail, I met promtail crash as below:

runtime/cgo pthread_create failed resource temporarily unavailable SIGABRT: abort

I think it is due to too many goroutines, which occupied all the resources.

Is there suggested maximum of the log files that promtail can tailing?

JStickler commented 3 days ago

Questions have a better chance of being answered if you ask them on the community forums.

webz8701 commented 3 days ago

Questions have a better chance of being answered if you ask them on the community forums.

Thank you, JStickler. I post this question at https://community.grafana.com/t/the-maximum-of-log-files-that-promtail-can-tailiing/137170