List of available External Metrics are showing up in Custom Metrics list and not External Metrics

[mattKaczorowski]

This one took some time to figure out. I was trying to debug if the adapter pod was able to provide external metrics, however when I check the external metrics endpoint, I see no resources.

$ curl http://localhost:8001/apis/external.metrics.k8s.io/v1beta1
{
  "kind": "APIResourceList",
  "apiVersion": "v1",
  "groupVersion": "external.metrics.k8s.io/v1beta1",
  "resources": []
}

However, the external metrics are being listed under the custom metrics endpoint

$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1" | jq 
{
  "kind": "APIResourceList",
  "apiVersion": "v1",
  "groupVersion": "custom.metrics.k8s.io/v1beta1",
  "resources": [
    {
      "name": "*/actions.googleapis.com|smarthome_action|local_event_count",
      "singularName": "",
      "namespaced": true,
      "kind": "MetricValueList",
      "verbs": [
        "get"
      ]
    },
    {
      "name": "*/actions.googleapis.com|smarthome_action|num_active_users",
      "singularName": "",
      "namespaced": true,
      "kind": "MetricValueList",
      "verbs": [
        "get"
      ]
    },

....

So I assumed that the external metrics were available under the custom.metrics.k8s.io endpoint. But when looking up metrics under the custom endpoint, it would cause the stackdriver adapter to panic.

After trying out a hunch, the external metrics worked as expected when changing to the external.metrics.k8s.io endpoint (as intended).

I am not sure if listing the external metrics under the custom endpoint is intentional, but if so, it is a bit misleading.

[mastersingh24]

Noticed the same thing and it appears as if list is not implemented for external metrics.

[nhha1602]

Hi @mattKaczorowski, How can you change to "external.metrics.k8s.io" endpoint?

Thank you

[CatherineF-dev]

HPA is still working though LIST is not implemented. This LIST increases memory a lot (17 Mi -> 58 Mi). https://github.com/GoogleCloudPlatform/k8s-stackdriver/pull/311

[BryceGust]

Would it be possible to send a message with this api call alerting users that this is not yet implemented? I spent about a week trying to stand up an hpa based on what I thought was an external metric, and most of my confusion came from not seeing any external metrics exposed.

I think some time could be saved even before actually implementing the LIST by printing something to stdout or embedding it the metadata of the response. Something like:

kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/" | jq .

INFO: LIST not implemented for external metrics. Currently an empty list is returned by default.
External metrics are still exposed. External metric names mirror custom metric names.
To access a specific metric try 'kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/*/<custom-metric-name>"
{
  "kind": "APIResourceList",
  "apiVersion": "v1",
  "groupVersion": "external.metrics.k8s.io/v1beta1",
  "resources": []
}

I don't have the most talent for being terse, but I believe that illustrates the idea.

This would have saved me a lot of time, and I came across a few stackoverflow questions with similar confusion and no answers that clear that confusion effectively.