[Question] Could jvm_exporter monitor Hbase?

[regardfs]

HI, guys: I am transferring zabbix stack to prometheus... I have red this article https://blog.godatadriven.com/hbase-prometheus-monitoring about how to monitor hbase by jmx_exporter, but It run as an agent. My hbase cluster managed by ambari, and it is hard to hack and hbase cluster in production could be restart. My question is does this jvm_export could pull metrics in already running hbase cluster? I saw this configuration of hostPort: 127.0.0.1:1234 and jmxUrl: service:jmx:rmi:///jndi/rmi://127.0.0.1:1234/jmxrmi How to config these in my scenario? Thanks a ton!!

[kelein]

Prepare Work, Download the jmx jar first: jmx_prometheus_javaagent-0.x.jar

To hadoop-2.7.3.tar.gz,hbase-1.2.4-bin.tar.gz, for example, use its installation package in the most simple configuration, in the local pseudo-distributed environment test;

Modify the hbase-1.2.4 / conf / hbase-env.sh to add the following configuration:

HBASE_OPTS = "$HBASE_OPTS -javaagent: /path_of_jar/jmx_prometheus_javaagent-0.7.jar=$port: /path_of_conf/hbase/hbase_jmx_config.yaml"

• Jmx_prometheus_javaagent-0.7.jar
• Port port: JMX Metrics Port
• Hbase_jmx_config.yaml: hbase metrics format configuration file

 Hbase_jmx_config.yaml format is as follows:

Rules:
  - pattern: Hadoop <service = HBase, name = RegionServer, sub = Regions> <> Namespace _ ([^ \ W _] +) _ table _ ([^ \ W _] +) _ region _ ([^ \ W _] +) _ metric _ (\ w + )
    Name: HBase_metric_ $ 4
    Labels:
      Namespace: "$ 1"
      Table: "$ 2"
      Region: "$ 3"
  - pattern: Hadoop <service = (\ w +), name = (\ w +), sub = (\ w +)> <> (\ w +)
    Name: HBase_ $ 2_ $ 3_ $ 4

Modify the above configuration file, boot hadoop and hbase one by one:

Start HDFS:

Hadoop-2.7.3/bin/hdfs namenode -format -force -nonInterActive
Hadoop-2.7.3/sbin/start-dfs.sh

If the startup fails, you may need to configure JAVA_HOME

Start HMaster:

Hbase-1.2.4/bin/ start-hbase.sh
Hbase-1.2.4/bin/ local-master-backup.sh start 1

Curl -i http: //localhost:7000/metrics

HBase_MetricsSystem_Stats_PublishAvgTime 0.0
HBase_MetricsSystem_Stats_NumActiveSources 4.0
HBase_Master_IPC_ResponseSize_75th_percentile 0.0
HBase_Master_IPC_RequestSize_min 0.0
HBase_Master_Server_averageLoad 0.0
...

Start Regionserver:

hbase-1.2.4/bin/local-regionservers.sh start 2 3 4

Port Define:

port	service
7000	HMaster JMX Metrics
7001	RegionServer JMX Metrics
7002	RegionServer JMX Metrics
7003	RegionServer JMX Metrics

curl -i http://localhost:7003/metrics

HBase_RegionServer_IPC_ResponseSize_95th_percentile 0.0
HBase_RegionServer_IPC_RequestSize_95th_percentile 0.0
HBase_MetricsSystem_Stats_PublishAvgTime 1.0
HBase_RegionServer_IPC_QueueCallTime_99th_percentile 0.0
HBase_RegionServer_IPC_numActiveHandler 1.0
HBase_RegionServer_IPC_QueueCallTime_mean 0.0
……

Finally, config the prometheus.yml as following：

global:
  scrape_interval: 5s
  external_labels:
    monitor: 'my-monitor'
scrape_configs:
  - job_name: 'hbase'
    static_configs:
      - targets: [
          '10.200.3.98:7000',
          '10.200.3.98:7001',
          '10.200.3.98:7002',
          '10.200.3.98:7003'
        ]

That's all !

[regardfs]

@Keleir Really appreciating your reply Thanks a ton!!!