I think the main problem here is juju_application is filtered as loki out of the box so it doesn't include message count from granfana-agent in other machines.
To Reproduce
deploy cos-lite
deploy a machine charm and relate it with grafana-agent to forward logs
Environment
App Version Status Scale Charm Channel Rev Address Exposed Message
alertmanager 0.27.0 active 1 alertmanager-k8s latest/edge 131 10.152.183.117 no
catalogue active 1 catalogue-k8s latest/edge 59 10.152.183.100 no
grafana 9.5.3 active 1 grafana-k8s latest/edge 118 10.152.183.166 no
loki 2.9.6 active 1 loki-k8s latest/edge 163 10.152.183.188 no
prometheus 2.52.0 active 1 prometheus-k8s latest/edge 211 10.152.183.219 no
traefik 2.11.0 active 1 traefik-k8s latest/edge 203 10.152.183.184 no Serving at 192.168.151.81
Bug Description
Similar topic with https://github.com/canonical/loki-k8s-operator/issues/435 but the issue as per the subject is still visible with latest/edge.
I think the main problem here is
juju_application
is filtered asloki
out of the box so it doesn't include message count from granfana-agent in other machines.To Reproduce
Environment
Relevant log output
Additional context
dashboard
out of the box
more raw data