Open nmiculinic opened 1 year ago
Currently the logdna_agent_fs_lines is a simple counter without any labels. For the purposes of observing the logging agent properly, it'd make things simpler to add additional labels to the metric, similar to promtail agent:
logdna_agent_fs_lines
Promtail include the filepath at least.
While I'd love the k8s enrichment to be included as well (pod, namespace, etc. in the labels of that metric), seeming like k8s things are being handled later in a middleware https://github.com/nmiculinic/logdna-agent-v2/blob/9b3763b10f5b893ad4cc634e4fc35db1ec973942/bin/src/_main.rs#L260 and isn't really present at the point of metric insertion: https://github.com/nmiculinic/logdna-agent-v2/blob/df491cd8cca74d98cc6d8ca7ce25d559073c602c/common/k8s/src/event_source.rs#L403
With some regex (e.g this is container log path):
/var/log/containers/<pod_name>_<namespace>-<random suffix>.log
it's possible in post-processing to determine k8s namespace of the log line being scrape
@nmiculinic
add additional labels to the metric, similar to promtail agent
in that link I see it adds some counters but not additional metadata.?
@dkhokhlov exactly
Currently the
logdna_agent_fs_lines
is a simple counter without any labels. For the purposes of observing the logging agent properly, it'd make things simpler to add additional labels to the metric, similar to promtail agent:Promtail include the filepath at least.
While I'd love the k8s enrichment to be included as well (pod, namespace, etc. in the labels of that metric), seeming like k8s things are being handled later in a middleware https://github.com/nmiculinic/logdna-agent-v2/blob/9b3763b10f5b893ad4cc634e4fc35db1ec973942/bin/src/_main.rs#L260 and isn't really present at the point of metric insertion: https://github.com/nmiculinic/logdna-agent-v2/blob/df491cd8cca74d98cc6d8ca7ce25d559073c602c/common/k8s/src/event_source.rs#L403
With some regex (e.g this is container log path):
it's possible in post-processing to determine k8s namespace of the log line being scrape