grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.97k stars 3.46k forks source link

Why does the pid keep growing when promtail is enabled with podman. #15048

Open Nemenisi opened 22 hours ago

Nemenisi commented 22 hours ago

You may submit questions in any of the following ways:

Start command : _podman run -d --network host --name promtail --privileged=true -v /var/grafana/promtail:/mnt/config -v /user/airflow/logs:/var/airflow_log localhost/grafana/promtail:3.2.0 -config.file=/mnt/config/promtail-config.yaml

Promtail-config.yaml: _server: http_listen_port: 9080 grpc_listen_port: 0 positions: filename: /tmp/positions1.yml ignore_invalid_yaml: true clients:

Then, use the command : systemctl status machine.slice Docs: man:systemd.special(7) Tasks: 1443 Memory: 11.3G CGroup: /machine.slice ├─libpod-2a5f9bf0ffe840b65d1aecb2fdc255b1a48d5acaf058e07291d0e4faa9776327.scope │ └─3889907 /usr/bin/loki -config.file=/mnt/config/loki-config.yaml ├─libpod-3e5c4498517c6ef0f08c1f07435b0a7d209d4aa3b8d12e3fc1cb1ec1e729344a.scope │ └─193271 /usr/bin/promtail -config.file=/mnt/config/promtail-config.yaml

And "cat /sys/fs/cgroup/pids/machine.slice/libpod-3e5c4498517c6ef0f08c1f07435b0a7d209d4aa3b8d12e3fc1cb1ec1e729344a.scope/pids.max" we get 2048

Then "cat /sys/fs/cgroup/pids/machine.slice/libpod-3e5c4498517c6ef0f08c1f07435b0a7d209d4aa3b8d12e3fc1cb1ec1e729344a.scope/pids.current

we get 1330.

**The problem is that when promtail is always forking new processes, the value of pids.current will increase and reach pids.max. After that, the system will stop the operation of Promtail.

What causes the value of pids.current to keep increasing? Can I limit its growth?**