Open stephen-dahl opened 5 months ago
Follow-up from this Slack thread.
I tried using
"{{workflow.labels.workflows.argoproj.io/cron-workflow}}"
received error
MetricsError: unable to substitute parameters for metric 'exec_count': failed to resolve {{=workflow.labels['workflows.argoproj.io/cron-workflow']}}, unable to substitute parameters for metric 'exec_duration': failed to resolve {{=workflow.labels['workflows.argoproj.io/cron-workflow']}}
value: "{{workflow.labels.workflows.argoproj.io/creator}}"
The description here seems to have copy+pasted a few different ones, so it's not exactly clear which variants you tried and which you didn't. All 3 of the above are different.
Workflow
(not CronWorkflow
)In the thread, @tico24 wrote that this works:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: wibblewobble-
spec:
metrics:
prometheus:
- name: exec_count
help: "Count of workflow execution by result status - workflow level"
labels:
- key: name
value: "{{workflow.labels.workflows.argoproj.io/creator}}"
- key: status
value: "{{workflow.status}}"
counter:
value: "1"
but not when using as a workflowSpec
of a CronWorkflow
. Tim's hypothesis there was: "My assumption is that the metrics parameters are unpacked before the cron label is applied to the running workflow, so it can't find the label - hence the error. I would agree that this is an issue."
expr
brackets syntaxI suggested using the expr
style brackets, i.e. {{=workflow.labels['workflows.argoproj.io/cron-workflow']}}
. Per the error it seems like that didn't work either, although not entirely clear due to copy+paste errors as per above
I'm also not sure if brackets are potentially affected by https://github.com/argoproj/argo-workflows/issues/11549
ya im all over the place, working on cleaning it up... sorry
the description has been updated with a new run ensuring consistency between all the parts
did some testing
{{workflow.labels.workflows.argoproj.io/creator}}
: worked{{workflow.labels.workflows.argoproj.io/cron-workflow}}
: worked{{=workflow.labels.workflows.argoproj.io/creator}}
: MetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression: unknown name creator (1:39) | workflow.labels.workflows.argoproj.io/creator | ......................................^
{{=workflow.labels.workflows.argoproj.io/cron-workflow}}
: MetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression: unknown name cron (1:39) | workflow.labels.workflows.argoproj.io/cron-workflow | ......................................^
{{=workflow.labels['workflows.argoproj.io/creator']}}
: MetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression "workflow.labels['workflows.argoproj.io/creator']"
{{=workflow.labels['workflows.argoproj.io/cron-workflow']}}
: MetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression "workflow.labels['workflows.argoproj.io/cron-workflow']"
the middle two shouldn't work, so those are correct.
the last two I would expect to work, so are buggy.
Regarding the first two -- those worked on a CronWorkflow
as well? Since Tim said it didn't work when he tried a Workflow
generated by a CronWorkflow
all of my testing was done by setting the metrics in the controller config workflow defaults, creating a cronworkflow, and submitting it through the UI
I have slightly similar issue to this one and https://github.com/argoproj/argo-workflows/issues/3989 when using a workflowTemplate
and a custom label in it.
Although the label monitoring
is set in the workflowTemplate
, it is not resolved. However, if I manually add this label in the UI as in the image below, it works.
It seems that the labels defined in the manifests are not being considered.
{{workflow.labels.workflows.argoproj.io/cron-workflow}}
: worked
In my case this didn't work, Argo v3.6.0-rc2
I am also encountering this issue using the expr syntax.
The log from the workflow-controller:
time="2024-11-07T10:48:08.354Z" level=error msg="unable to substitute parameters for metric 'workflow_completed_count': failed to evaluate expression \" workflow.labels[\\\\\\\"example.com/environment-name\\\\\\\"] \"" namespace=workspace-workflow-reference workflow=workflow-reference-s9475
The expression:
value: '{{= workflow.labels["example.com/environment-name"] }}'
I tested the expression on expr Playground on this data and it worked as expected:
workflow:
labels:
example.com/environment-name: "foo"
For those looking for immediate solutions, the workaround which works for me is to use workflow.labels.json
along with fromJSON()
from expr.
value: '{{= fromJSON(workflow.labels.json)["example.com/environment-name"] }}'
Another thing to note is that labels on the workflowTemplate
referenced at spec.workflowTemplateRef
of the running workflow are not transferred to the workflow
resource. So for this to work you need to make sure you set the labels on the workflow as well by setting them via the spec.workflowMetadata.labels
field on the workflowTemplate
.
Pre-requisites
:latest
image tag (i.e.quay.io/argoproj/workflow-controller:latest
) and can confirm the issue still exists on:latest
. If not, I have explained why, in detail, in my description below.What happened/what did you expect to happen?
https://cloud-native.slack.com/archives/C01QW9QSSSK/p1717447990226269
I tried using
"{{=workflow.labels['workflows.argoproj.io/cron-workflow']}}"
as a metric label inside a workflow created by a cron and received errorMetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression "workflow.labels['workflows.argoproj.io/cron-workflow']"
Version
latest
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
Logs from the workflow controller
Logs from in your workflow's wait container