search

zilliztech / milvus-operator

The Kubernetes Operator of Milvus.
https://milvus.io
Apache License 2.0
34 stars 20 forks source link

is grafana available on the operator? #40

Closed webcoderz closed 7 months ago

webcoderz commented 8 months ago

can we enable grafana with the operator or cluster?

haorenfsa commented 8 months ago

We haven't released a formal grafana dashboard for milvus-operator yet, but you can find the dashboard for milvus cluster in milvus repo: https://github.com/milvus-io/milvus/tree/master/deployments/monitor/grafana

haorenfsa commented 8 months ago

The metric export is by default enabled, for milvus in port 9091, and for milvus-operator in port 8080.

webcoderz commented 8 months ago

okay thanks i see them active, but doesnt seem like the kube prom stack with prometheus operator is able to find the metrics

haorenfsa commented 8 months ago

you mean metrics of milvus or operator, or both of them?

webcoderz commented 8 months ago

Yea both, I was able to port forward and see the metrics functional but the prometheus operator didn't find them when I added the dashboard

haorenfsa commented 8 months ago

under your milvus' namespace, could you find the PodMonitor created by milvus-operator? and which version of prometheus operator are you using?

webcoderz commented 8 months ago

Just saw this Will check tomorrow when I am back in front of my cluster

webcoderz commented 8 months ago

Okay seems like ArgoCD is having trouble with the operator and it's stuck in progressing , I have to delete and redeploy or something

webcoderz commented 8 months ago

Got it working.. the argoCD was having issues so once I reset metrics came up. Thanks! 🙏

webcoderz commented 8 months ago

okay so i just changed form milvus standalone to milvus cluster, and same issue, i port forward and can see the metrics on both the operator and the cluster, and i load in the json dashboard and it cant pick up the metrics from the services