add workflow labels to metrics?

[linkingli]

Summary

add workflow labels to the metric?

Use Cases

When I have two workflowTemplates (A,B), I want to know the overall situation of workflow of A and B respectively. The current metric does not support me to do this. Since there is no label for workflow.

NOW:

argo_workflows_count{container="workflow-controller", endpoint="metrics", 
instance="127.0.0.1:9090", job="workflow-controller-metrics", namespace="argo",
 pod="workflow-controller-7d69c4b564-t6xn8",
 service="workflow-controller-metrics", status="Error"}

BETTER:

argo_workflows_count{container="workflow-controller", endpoint="metrics", 
instance="127.0.0.1:9090", job="workflow-controller-metrics", namespace="argo",
 pod="workflow-controller-7d69c4b564-t6xn8",
 service="workflow-controller-metrics", status="Error",labelName="LabelValue"}

---

Message from the maintainers:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

[LilTwo]

Hi @terrytangyuan , is this a considerable enhancement? If so, I can do some studies about it, currently there is no way to know pending workflow count since even with custom metrics, that is very inconvenient if you run many kinds of workflows in your cluster

[terrytangyuan]

Yeah feel free to submit a proposal

[moonyoungCHAE]

I found it can be work on custom metrics. And I'm worried about that if workflow name is added on default metric labels, it can be create huge amount of metrics.

[LilTwo]

Hi @terrytangyuan opened a PR, please take a look if you have time, thanks!

[tooptoop4]

k8s pod metrics (not argo) include pod name, so seems reasonable to include workflow name