search

IBM / ibm-spectrum-scale-bridge-for-grafana

This tool allows the IBM Storage Scale users to perform performance monitoring for IBM Storage Scale devices using third-party applications such as Grafana or Prometheus software.
Apache License 2.0
30 stars 17 forks source link

cannot get metadata from pmcollector #199

Closed liaozexian1204 closed 6 months ago

liaozexian1204 commented 6 months ago

when starting this zimonGrafanaIntf.py application, it reports below and exits after retry "QueryHandler: getTopology returns no data", the server respond is "503 Connection refused from server"

I checked gpfs gui and pmcollectors are active and running by "systemctl status gpfsgui" and "systemctl status pmcollector" and zimonGrafanaIntf.py is started at the same host with gpfs gui and pmcollector.

Is there something I missed with the configuration? what is the serverPort in "GPFS Server" section of config.ini. Is there any CLI that I can check the listening port of queries on pmcollector?

I also find the query Rest APIs are /sysmon/v1/perfmon/topo, but this API can't be found in GPFS doc https://www.ibm.com/docs/en/storage-scale/5.1.9?topic=endpoints-performance-monitoring

liaozexian1204 commented 6 months ago

my gpfs version is 5.1.6.1, grafana_bridge is v7.1 I got below config in pmcollector config: fallbackqueryinterface = "0.0.0.0" fallbackqueryport = "9084" and bridge config.ini server port is also 9084, but cannot connect checked firewall is disabled

ps20renar commented 6 months ago

Hallo liaozexian1204, for installation make at first a mmperfmon config show --apikey scale_grafana Put this key in the config.ini on the key parameter and restart the bridge.

liaozexian1204 commented 6 months ago

apikey of scale_grafana is already configured, but still cannot connect. grafana bridge reports the same error: 503 connection refused. in pmcollector log, it says "ServiceInterface: read 1error: closing socket 127.0.0.1"

any thoughts?

Helene commented 6 months ago

apikey of scale_grafana is already configured, but still cannot connect. grafana bridge reports the same error: 503 connection refused. in pmcollector log, it says "ServiceInterface: read 1error: closing socket 127.0.0.1"

any thoughts?

on the pmcollector node please execute the following commands and put the output here

# mmlscluster

# mmhealth node show perfmon -v

# mmsysmonc noderoles list
liaozexian1204 commented 6 months ago

image

image image the collector node is gpfs01 in my cluster

Helene commented 6 months ago

Are you running the grafana-bridge directly on gpfs01 or on a remote host? I need probably review your config.ini How do you start the grafana-bridge?

liaozexian1204 commented 6 months ago

I think I found the reason - indeed the configuration is incorrect. pmcollector config: (as in https://www.ibm.com/docs/en/spectrum-control/5.4.4?topic=systems-configuring-collection-performance-data-spectrum-scale) fallbackqueryinterface="0.0.0.0" fallbackqueryport="29980"

grafana-bridge config.ini: serverPort = 29980

after I changed config.ini "serverPort" back to 9980, pmcollector metadata can be retrieved. It seems that fallbackqueryport is used for other purpose, not for zimon rest api.

thanks, I'm closing this ticket.

Helene commented 6 months ago

I think I found the reason - indeed the configuration is incorrect. pmcollector config: (as in https://www.ibm.com/docs/en/spectrum-control/5.4.4?topic=systems-configuring-collection-performance-data-spectrum-scale) fallbackqueryinterface="0.0.0.0" fallbackqueryport="29980"

grafana-bridge config.ini: serverPort = 29980

after I changed config.ini "serverPort" back to 9980, pmcollector metadata can be retrieved. It seems that fallbackqueryport is used for other purpose, not for zimon rest api.

thanks, I'm closing this ticket.

Oh, you are using perfmon for Spectrum Control, which requires specific port configuration. Probably I need to document this case. Thanks for your feedback.