`dashboards 404 Not Found` - "No metrics to display"

[harrywm]

Hello!

I've gone about installing the extension to allow our developers an (experimental) "single pane of glass" view of their deployments. I'm running into an issue where the metrics tab simply displays "No metrics to display", where, on closer inspection, the dashboards request to is returning a 404 when trying to pull in what I assume is a JSON blob for the applicable graphs.

The installation I've gone about setting up is as follows, using the argo-extension-installer with the Argo Helm charts, and the manifests included in this repository.

ArgoCD Helm values:

Metrics-sever manifests (altered to use a more representative name for our cluster) - note, i've tried with both TLS enabled and disabled: Deployment,

Service,

Would appreciate some help as to where we're going wrong here. I've port-forwarded the metrics-server service locally, and pinged a representative endpoint (metrics/api/applications/reloader/groupkinds/pod/dashboards) and got JSON in return, so I'm assuming it's connectivity between Argo and the Metrics-Server, or some misconfiguration of a URL somewhere.

[mcanevet]

Same issue here. I can see this error message in argocd-server's logs: "read message err: websocket: close 1001 (going away)" or "read message err: websocket: close 1005 (no status)". Interesting fact: when I remove the service in front of the argocd-metrics-server, I don't get this messages, so I guess argocd-server actually tries to communicate with argocd-metrics-server, but somehow the websocket is closed.

[ofirshaya]

@leoluz any chance you can take a look?

[leoluz]

404 can be caused by 2 things:

1. Proxy extension isn't enabled in your Argo CD API server. In order to do so please follow this document: https://argo-cd.readthedocs.io/en/stable/developer-guide/extensions/proxy-extensions/#configuration
2. Argo CD metrics server is not configured properly.

I suspect that this is caused by #1 so please, ensure that the feature flag is enabled. If the proxy extension feature is enabled you should be able to see the following entry in the API Server logs: Registering extensions...

[harrywm]

404 can be caused by 2 things:

1. Proxy extension isn't enabled in your Argo CD API server. In order to do so please follow this document: https://argo-cd.readthedocs.io/en/stable/developer-guide/extensions/proxy-extensions/#configuration
2. Argo CD metrics server is not configured properly.

I suspect that this is caused by #1 so please, ensure that the feature flag is enabled. If the proxy extension feature is enabled you should be able to see the following entry in the API Server logs: Registering extensions...

Thanks @leoluz !

I'll go about enabling it now. I assume I'll need to setup some proxy config (as mentioned in the docs), are there any examples of this anywhere so I'm not shooting in the dark?

[leoluz]

@harrywm Correct. You need this configuration in the argocd-cm

  extension.config: |-
    extensions:
      - name: metrics
        backend:
          services:
            - url: <METRICS_SERVER_URL>

And you need to authorize this extension in the argocd-rbac-cm:

p, role:readonly, extensions, invoke, metrics, allow

I will update the main docs in this repo to help future users about the required steps. Tks for pointing this out!

[harrywm]

Thanks so much @leoluz! Happy to review some docs changes as well, if you need any look over them from fresh eyes :)

[leoluz]

@harrywm here it is: https://github.com/argoproj-labs/argocd-extension-metrics/pull/64/files

[ofirshaya]

I can confirm this works after the added information from @leoluz

thank you!

just a disclaimer that the url should be along the lines of

http://YOUR-INTERNAL-SVC-ADDRESS:9003