Closed regardfs closed 7 years ago
Prepare Work, Download the jmx jar first: jmx_prometheus_javaagent-0.x.jar
To hadoop-2.7.3.tar.gz
,hbase-1.2.4-bin.tar.gz
, for example, use its installation package in the most simple configuration, in the local pseudo-distributed environment test;
Modify the hbase-1.2.4 / conf / hbase-env.sh
to add the following configuration:
HBASE_OPTS = "$HBASE_OPTS -javaagent: /path_of_jar/jmx_prometheus_javaagent-0.7.jar=$port: /path_of_conf/hbase/hbase_jmx_config.yaml"
Hbase_jmx_config.yaml
format is as follows:
Rules:
- pattern: Hadoop <service = HBase, name = RegionServer, sub = Regions> <> Namespace _ ([^ \ W _] +) _ table _ ([^ \ W _] +) _ region _ ([^ \ W _] +) _ metric _ (\ w + )
Name: HBase_metric_ $ 4
Labels:
Namespace: "$ 1"
Table: "$ 2"
Region: "$ 3"
- pattern: Hadoop <service = (\ w +), name = (\ w +), sub = (\ w +)> <> (\ w +)
Name: HBase_ $ 2_ $ 3_ $ 4
Modify the above configuration file, boot hadoop and hbase one by one:
Hadoop-2.7.3/bin/hdfs namenode -format -force -nonInterActive
Hadoop-2.7.3/sbin/start-dfs.sh
If the startup fails, you may need to configure JAVA_HOME
Hbase-1.2.4/bin/ start-hbase.sh
Hbase-1.2.4/bin/ local-master-backup.sh start 1
Curl -i http: //localhost:7000/metrics
HBase_MetricsSystem_Stats_PublishAvgTime 0.0
HBase_MetricsSystem_Stats_NumActiveSources 4.0
HBase_Master_IPC_ResponseSize_75th_percentile 0.0
HBase_Master_IPC_RequestSize_min 0.0
HBase_Master_Server_averageLoad 0.0
...
hbase-1.2.4/bin/local-regionservers.sh start 2 3 4
port | service |
---|---|
7000 | HMaster JMX Metrics |
7001 | RegionServer JMX Metrics |
7002 | RegionServer JMX Metrics |
7003 | RegionServer JMX Metrics |
curl -i http://localhost:7003/metrics
HBase_RegionServer_IPC_ResponseSize_95th_percentile 0.0
HBase_RegionServer_IPC_RequestSize_95th_percentile 0.0
HBase_MetricsSystem_Stats_PublishAvgTime 1.0
HBase_RegionServer_IPC_QueueCallTime_99th_percentile 0.0
HBase_RegionServer_IPC_numActiveHandler 1.0
HBase_RegionServer_IPC_QueueCallTime_mean 0.0
……
Finally, config the prometheus.yml
as following:
global:
scrape_interval: 5s
external_labels:
monitor: 'my-monitor'
scrape_configs:
- job_name: 'hbase'
static_configs:
- targets: [
'10.200.3.98:7000',
'10.200.3.98:7001',
'10.200.3.98:7002',
'10.200.3.98:7003'
]
That's all !
@Keleir Really appreciating your reply Thanks a ton!!!
HI, guys: I am transferring zabbix stack to prometheus... I have red this article https://blog.godatadriven.com/hbase-prometheus-monitoring about how to monitor hbase by jmx_exporter, but It run as an agent. My hbase cluster managed by ambari, and it is hard to hack and hbase cluster in production could be restart. My question is does this jvm_export could pull metrics in already running hbase cluster? I saw this configuration of
hostPort: 127.0.0.1:1234
andjmxUrl: service:jmx:rmi:///jndi/rmi://127.0.0.1:1234/jmxrmi
How to config these in my scenario? Thanks a ton!!