search

dabealu / zookeeper-exporter

zookeeper exporter for prometheus
MIT License
139 stars 74 forks source link

Prometheus scrape error: expected value after metric, got "MNAME" #34

Closed glongzh closed 3 years ago

glongzh commented 3 years ago

Hi, i deploy a zookeeper cluster with this operator, zookeeper version is 3.6.1, and mntr command is in the 4lw.commands.whitelist. After i deploy the exporter, prometheus got an error: expected value after metric, got "MNAME", when i check the metrics endpoint, i got something like this:

zk_concurrent_request_processing_in_commit_processor_count{zk_host="zookeeper-1.zookeeper-headless:2181"} 0.0
zk_write_per_namespace_count{key="config"}{zk_host="zookeeper-1.zookeeper-headless:2181"} 9.0
zk_jvm_memory_pool_bytes_max{pool="Compressed{zk_host="zookeeper-0.zookeeper-headless:2181"} Class
zk_jvm_threads_state{state="RUNNABLE"}{zk_host="zookeeper-0.zookeeper-headless:2181"} 15.0
zk_quorum_ack_latency{quantile="0.99"}{zk_host="zookeeper-0.zookeeper-headless:2181"} NaN
zk_local_write_committed_time_ms_sum{zk_host="zookeeper-0.zookeeper-headless:2181"} 770.0
zk_jvm_memory_bytes_init{area="nonheap"}{zk_host="zookeeper-0.zookeeper-headless:2181"} 7667712.0
zk_write_per_namespace{key="log_dir_event_notification",quantile="0.5"}{zk_host="zookeeper-1.zookeeper-headless:218
1"} NaN
zk_om_proposal_process_time_ms_count{zk_host="zookeeper-2.zookeeper-headless:2181"} 0.0
zk_write_per_namespace_sum{key="cluster"}{zk_host="zookeeper-2.zookeeper-headless:2181"} 64.0
zk_proposal_latency_count{zk_host="zookeeper-0.zookeeper-headless:2181"} 0.0
zk_jvm_buffer_pool_used_bytes{pool="mapped"}{zk_host="zookeeper-1.zookeeper-headless:2181"} 0.0
zk_write_per_namespace{key="controller",quantile="0.5"}{zk_host="zookeeper-1.zookeeper-headless:2181"} NaN
zk_read_per_namespace_sum{key="cluster"}{zk_host="zookeeper-0.zookeeper-headless:2181"} 124.0
zk_write_per_namespace{key="latest_producer_id_block",quantile="0.5"}{zk_host="zookeeper-0.zookeeper-headless:2181"
} NaN
zk_write_commitproc_time_ms{quantile="0.5"}{zk_host="zookeeper-1.zookeeper-headless:2181"} NaN
zk_jvm_memory_bytes_committed{area="heap"}{zk_host="zookeeper-2.zookeeper-headless:2181"} 2.51396096E8
zk_prep_processor_queue_time_ms_count{zk_host="zookeeper-0.zookeeper-headless:2181"} 179132.0
zk_jvm_memory_pool_bytes_used{pool="Tenured{zk_host="zookeeper-0.zookeeper-headless:2181"} Gen"}
zk_startup_txns_load_time_sum{zk_host="zookeeper-1.zookeeper-headless:2181"} 0.0
zk_last_client_response_size{zk_host="zookeeper-0.zookeeper-headless:2181"} 16.0
zk_commit_propagation_latency_sum{zk_host="zookeeper-0.zookeeper-headless:2181"} 0.0
zk_num_alive_connections{zk_host="zookeeper-1.zookeeper-headless:2181"} 1.0
zk_process_max_fds{zk_host="zookeeper-1.zookeeper-headless:2181"} 1048576.0
zk_max_file_descriptor_count{zk_host="zookeeper-1.zookeeper-headless:2181"} 1048576.0
zk_jvm_memory_pool_allocated_bytes_total{pool="Compressed{zk_host="zookeeper-0.zookeeper-headless:2181"} Class
zk_response_packet_cache_hits{zk_host="zookeeper-0.zookeeper-headless:2181"} 96.0
zk_read_commit_proc_req_queued_count{zk_host="zookeeper-0.zookeeper-headless:2181"} 179230.0
zk_reads_after_write_in_session_queue_sum{zk_host="zookeeper-0.zookeeper-headless:2181"} 1.0
zk_jvm_buffer_pool_capacity_bytes{pool="direct"}{zk_host="zookeeper-1.zookeeper-headless:2181"} 147861.0

I think the problem is the "Class" value of zk_jvm_memory_pool_bytes_max metric, is that right? Is this a bug or i misconfigured something? Thanks.

dabealu commented 3 years ago

Hey @glongzh, sorry for late reply. It's a bug and should be fixed starting from v0.1.12 of the exporter, thanks to @thinker0

PRs: https://github.com/dabealu/zookeeper-exporter/pull/37 https://github.com/dabealu/zookeeper-exporter/pull/38