search

kpetremann / mqtt-exporter

Simple generic MQTT Prometheus exporter for IoT working out of the box
https://hub.docker.com/r/kpetrem/mqtt-exporter
MIT License
108 stars 30 forks source link

exporter does not produce any device metric #25

Closed runningman84 closed 2 years ago

runningman84 commented 2 years ago

I have tried the exporter in my k8s setup. But it does not produce any device metrics.

This is the zigbee2mqtt log:

Info 2022-05-26 21:58:11MQTT publish: topic 'zigbee2mqtt/living_room/tv_left/plug', payload '{"current":0.11,"energy":287.04,"last_seen":"2022-05-26T19:58:11.525Z","linkquality":162,"power":17,"state":"ON","voltage":239}'
Info 2022-05-26 21:58:17MQTT publish: topic 'zigbee2mqtt/living_room/cabinet/light', payload '{"last_seen":"2022-05-26T19:58:17.361Z","linkquality":27,"state":"ON","update":{"state":"idle"},"update_available":false}'
Info 2022-05-26 21:58:17MQTT publish: topic 'zigbee2mqtt/living_room/tv_left/plug', payload '{"current":0.11,"energy":287.04,"last_seen":"2022-05-26T19:58:17.519Z","linkquality":162,"power":17,"state":"ON","voltage":239}'

This is the container log:

INFO:mqtt-exporter:listening to "zigbee2mqtt"

This is the container output:

/ # export
export CHART_1653595418_MQTT_EXPORTER_PORT='tcp://10.43.36.6:9000'
export CHART_1653595418_MQTT_EXPORTER_PORT_9000_TCP='tcp://10.43.36.6:9000'
export CHART_1653595418_MQTT_EXPORTER_PORT_9000_TCP_ADDR='10.43.36.6'
export CHART_1653595418_MQTT_EXPORTER_PORT_9000_TCP_PORT='9000'
export CHART_1653595418_MQTT_EXPORTER_PORT_9000_TCP_PROTO='tcp'
export CHART_1653595418_MQTT_EXPORTER_SERVICE_HOST='10.43.36.6'
export CHART_1653595418_MQTT_EXPORTER_SERVICE_PORT='9000'
export CHART_1653595418_MQTT_EXPORTER_SERVICE_PORT_METRICS='9000'
export GPG_KEY='E3FF2839C048B25C084DEBE9B26995E310250568'
export HOME='/root'
export HOSTNAME='chart-1653595418-mqtt-exporter-855c5d668d-qdcrg'
export KUBERNETES_PORT='tcp://10.43.0.1:443'
export KUBERNETES_PORT_443_TCP='tcp://10.43.0.1:443'
export KUBERNETES_PORT_443_TCP_ADDR='10.43.0.1'
export KUBERNETES_PORT_443_TCP_PORT='443'
export KUBERNETES_PORT_443_TCP_PROTO='tcp'
export KUBERNETES_SERVICE_HOST='10.43.0.1'
export KUBERNETES_SERVICE_PORT='443'
export KUBERNETES_SERVICE_PORT_HTTPS='443'
export LANG='C.UTF-8'
export LOG_LEVEL='DEBUG'
export MQTT_ADDRESS='mosquitto.hass.svc.cluster.local'
export MQTT_KEEPALIVE='60'
export MQTT_PORT='1883'
export MQTT_TOPIC='zigbee2mqtt'
export PATH='/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
export PROMETHEUS_PORT='9000'
export PROMETHEUS_PREFIX='mqtt_'
export PWD='/'
export PYTHON_GET_PIP_SHA256='e235c437e5c7d7524fbce3880ca39b917a73dc565e0c813465b7a7a329bb279a'
export PYTHON_GET_PIP_URL='https://github.com/pypa/get-pip/raw/38e54e5de07c66e875c11a1ebbdb938854625dd8/public/get-pip.py'
export PYTHON_PIP_VERSION='22.0.4'
export PYTHON_SETUPTOOLS_VERSION='58.1.0'
export PYTHON_VERSION='3.9.12'
export SHLVL='2'
export TERM='xterm'
export TOPIC_LABEL='topic'
export TZ='UTC'
export ZIGBEE2MQTT_AVAILABILITY='True'
/ # cd /tmp/
/tmp # wget 127.0.0.1:9000
Connecting to 127.0.0.1:9000 (127.0.0.1:9000)
saving to 'index.html'
index.html           100% |**********************************************************************************************************************************************************************************************|  1980  0:00:00 ETA
'index.html' saved
/tmp # cat index.html 
# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 347.0
python_gc_objects_collected_total{generation="1"} 7.0
python_gc_objects_collected_total{generation="2"} 0.0
# HELP python_gc_objects_uncollectable_total Uncollectable object found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
# HELP python_gc_collections_total Number of times this generation was collected
# TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 42.0
python_gc_collections_total{generation="1"} 3.0
python_gc_collections_total{generation="2"} 0.0
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="9",patchlevel="12",version="3.9.12"} 1.0
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 2.3732224e+07
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 1.8264064e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.65359542076e+09
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.28
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 7.0
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1.048576e+06
# HELP mqtt_message_total Counter of received messages
# TYPE mqtt_message_total counter

Btw. I have created a helm chart for your project: https://github.com/k8s-at-home/charts/pull/1584

kpetremann commented 2 years ago

Hello,

Weird. There is nothing wrong with your payload nor your topic.

I've tried a quick test and parsing is working:

 topic = "zigbee2mqtt/living_room/tv_left/plug"
    payload = '{"current":0.11,"energy":287.04,"last_seen":"2022-05-26T19:58:11.525Z","linkquality":162,"power":17,"state":"ON","voltage":239}'

    parsed_topic, parsed_payload = _parse_message(topic, payload)

    assert parsed_topic == "zigbee2mqtt_living_room_tv_left_plug"
    assert parsed_payload == {
        "current": 0.11,
        "energy": 287.04,
        "last_seen": "2022-05-26T19:58:11.525Z",
        "linkquality": 162,
        "power": 17,
        "state": "ON",
        "voltage": 239,
    }

    _parse_metrics(parsed_payload, parsed_topic, "zigbee2mqtt_")

    rich.print(parsed_payload)
    rich.print(prom_metrics)

Result :

pytest -vvv -s tests/functional/test_parse_message.py::test_parse_message_issue

tests/functional/test_parse_message.py::test_parse_message_issue {'current': 0.11, 'energy': 287.04, 'last_seen': '2022-05-26T19:58:11.525Z', 'linkquality': 162, 'power': 17, 'state': 'ON', 'voltage': 239}
{
    'mqtt_zigbee2mqtt_current': prometheus_client.metrics.Gauge(mqtt_zigbee2mqtt_current),
    'mqtt_zigbee2mqtt_energy': prometheus_client.metrics.Gauge(mqtt_zigbee2mqtt_energy),
    'mqtt_zigbee2mqtt_linkquality': prometheus_client.metrics.Gauge(mqtt_zigbee2mqtt_linkquality),
    'mqtt_zigbee2mqtt_power': prometheus_client.metrics.Gauge(mqtt_zigbee2mqtt_power),
    'mqtt_zigbee2mqtt_state': prometheus_client.metrics.Gauge(mqtt_zigbee2mqtt_state),
    'mqtt_zigbee2mqtt_voltage': prometheus_client.metrics.Gauge(mqtt_zigbee2mqtt_voltage)
}
PASSED

So here my questions for you:

Btw. I have created a helm chart for your project

Nice! thanks

kpetremann commented 2 years ago

Please try with this commit I pushed in master branch: https://github.com/kpetremann/mqtt-exporter/commit/a652f202c259b3bc5a6207e9ddfa7b76374c4591

It will provide the reason of connection failure if any.

runningman84 commented 2 years ago

Some additional hints...

I see this log in the mosquitto broker:

mosquitto 1653650049: New client connected from 10.42.0.229 as auto-C07CCC48-4D94-E384-CB31-3F1CFF248452 (p2, c1, k60).

Even in debug mode there is only the single log line:

INFO:mqtt-exporter:listening to "zigbee2mqtt"

Auth is disabled in mosquitto.

I will try your latest image later this afternoon.

kpetremann commented 2 years ago

oh I did not see you set MQTT_TOPIC to zigbee2mqtt. With this, you only subscribe to the exact topic zigbee2mqtt. It does not match with the different topics from zigbee2mqtt: topic 'zigbee2mqtt/living_room/tv_left/plug', etc...

Try with: MQTT_TOPIC="zigbee2mqtt/#"

runningman84 commented 2 years ago

perfect, I changed to topic as suggest and it seems to work fine now!