some label queries don't work

[bribroder]

An 'app' label query like this show several apps:

$ kubectl cost label -l 'app'

+---------+--------------------+--------------------+-----------------+
| CLUSTER | LABEL:APP          | MONTHLY RATE (ALL) | COST EFFICIENCY |
+---------+--------------------+--------------------+-----------------+
| asdf    | __unallocated__    |         1234.00 |        0.73311 |
|         | cert-manager       |           0.000000 |        0.000000 |
|         | webhook            |           0.000000 |        0.000000 |
|         | prometheus         |           0.000000 |        0.000000 |
+---------+--------------------+--------------------+-----------------+
| SUMMED  |                    |        1234.00 |                 |
+---------+--------------------+--------------------+-----------------+

But some other label queries group everything under unallocated even though there are pods with those labels:

$ kubectl cost label -l 'app.kubernetes.io/name'

+---------+------------------------------+--------------------+-----------------+
| CLUSTER | LABEL:APP.KUBERNETES.IO/NAME | MONTHLY RATE (ALL) | COST EFFICIENCY |
+---------+------------------------------+--------------------+-----------------+
| asdf   | __unallocated__              |        1234.00 |        0.73311 |
+---------+------------------------------+--------------------+-----------------+
| SUMMED  |                              |        1234.00 |                 |
+---------+------------------------------+--------------------+-----------------+

I get the same result for the label 'name'--is there some trick to the syntax?

[dwbrown2]

Looks like a potential label sanitization issue. Were you able to see if other labels were available?

[bribroder]

label 'component' did return results, but that's the only other label i've found that works--lots of other labels don't return anything:

• k8s-app
• name
• release
• pod-template-generation
• heritage
• chart

[michaelmdresser]

I can't address this directly right now, but the likely workaround is to replace most punctuation-like characters with _. Try this:

kubectl cost label -l 'app_kubernetes_io_name'

For explanation, and if someone wants to pick this up: Generally this is caused by Prometheus label sanitization. To resolve this issue, we should probably improve Kubecost's handling of queries like this to avoid confusion, but a less-good workaround is to implement the sanitization in kubectl cost itself.

[bribroder]

Thank you for the suggestion! I experimented with different character substitutions but couldn't find any that worked. This hypothesis also doesn't address all the unpunctuated examples above, such as 'name', 'release', 'chart', etc

[michaelmdresser]

That's quite odd! I'm able to see different results for kubectl cost label -l 'app_kubernetes_io_name' and kubectl cost label -l 'chart' in my environment, on v0.6.0 of kubectl-cost.

The next step in debugging is to check your Kubecost data, because kubectl cost label is served by the Kubecost Allocation API. In the Kubecost UI, are you able to aggregate and/or filter by the labels you mentioned? name, release, chart, etc.? Alternatively, you can run a /model/allocation query (unaggregated, please) and see if the labels show up on the workloads you expect them to.

[bribroder]

Ahh OK, I think this is likely an issue with the underlying API and nothing to do with the kubectl cost utility--thank you for the debugging