When grafana-agent is scraping log folders with hundreds of thousands of files it creates thousands of threads and consumes a lot of CPU time. It would be desirable to be able to customize the operator so that some log folders can be excluded (using the __path_exclude__ directive for example).
This problem was observed in particular with the /var/log/libvirt/qemu/ folder. There is a related enhancement proposal in loki (canonical/loki-k8s-operator#439) regarding the huge cardinality of the scraped data that might be tackled together with this one.
Enhancement Proposal
When grafana-agent is scraping log folders with hundreds of thousands of files it creates thousands of threads and consumes a lot of CPU time. It would be desirable to be able to customize the operator so that some log folders can be excluded (using the
__path_exclude__directive for example). This problem was observed in particular with the/var/log/libvirt/qemu/folder. There is a related enhancement proposal in loki (canonical/loki-k8s-operator#439) regarding the huge cardinality of the scraped data that might be tackled together with this one.