skureal
commented
1 year ago Hi @riodda
I got stuck in my setup at exactly were you are and it took me while to figure it out.
In my case I set up "Shelly Plus 1" with "Plus 1 AddON" and multiple "DS18B20" temperature sensors.
On my Ubuntu server which runs the prometheus server I installed mosquitto:
sudo apt install mosquitto mosquitto-clientsI created a user and password:
sudo mosquitto_passwd -c /etc/mosquitto/passwd usernameThen I configured the "Shelly Plus 1" to send RPC status notifications over MQTT and Generic status update over MQTT to the mosquitto service with the user & password just generated before.
As next step the data import to mosquitto was checked (for 1 sensor with id=100)
mosquitto_sub -t "shellyplus1-7c87ce63680c/status/temperature:100" -u username -P password -v
shellyplus1-7c87ce63680c/status/temperature:100 {"id": 100,"tC":43.3, "tF":110.0}
shellyplus1-7c87ce63680c/status/temperature:100 {"id": 100,"tC":43.4, "tF":110.2}
shellyplus1-7c87ce63680c/status/temperature:100 {"id": 100,"tC":43.6, "tF":110.4}
shellyplus1-7c87ce63680c/status/temperature:100 {"id": 100,"tC":43.7, "tF":110.6}The value of interest is Temperature in Celsius tC which is renamed to Heizung (german for heater).
The config.yaml in my case:
mqtt:
# The MQTT broker to connect to
server: tcp://MYIPADDRESS:MYPORT
# Optional: Username and Password for authenticating with the MQTT Server
user: username
password: password
# The Topic path to subscribe to. Be aware that you have to specify the wildcard.
topic_path: shellyplus1-7c87ce63680c/status/temperature:100
# The MQTT QoS level
qos: 0
cache:
# Timeout. Each received metric will be presented for this time if no update is send via MQTT.
# Set the timeout to -1 to disable the deletion of metrics from the cache. The exporter presents the ingest timestamp
# to prometheus.
timeout: 24h
json_parsing:
# Separator. Used to split path to elements when accessing json fields.
# You can access json fields with dots in it. F.E. {"key.name": {"nested": "value"}}
# Just set separator to -> and use key.name->nested as mqtt_name
separator: .
# This is a list of valid metrics. Only metrics listed here will be exported
metrics:
# The name of the metric in prometheus
- prom_name: heizung
# The name of the metric in a MQTT JSON message
mqtt_name: tC
# The prometheus help text for this metric
help: temperature reading
# The prometheus type for this metric. Valid values are: "gauge" and "counter"
type: gauge
# A map of string to string for constant labels. This labels will be attached to every prometheus metric
const_labels:
sensor_type: heizung
# The name of the metric in prometheus
Then I started mqtt2prometheus with the config.yaml in same folder mqtt2prometheus/bin
./mqtt2prometheus.linux_amd64 -config config.yaml -log-level debug
2023-02-15T01:52:36+01:00 info mqttclient/mqttClient.go:20 Connected to MQTT Broker
2023-02-15T01:52:36+01:00 info mqttclient/mqttClient.go:21 Will subscribe to topic {"topic": "shellyplus1-7c87ce63680c/status/temperature:100"}
2023-02-15T01:52:36+01:00 debug web/tls_config.go:195 {"level": "info", "msg": "TLS is disabled.", "http2": false}
2023-02-15T01:52:41+01:00 debug metrics/ingest.go:42 Got message {"topic": "shellyplus1-7c87ce63680c/status/temperature:100", "payload": "{\"id\": 100,\"tC\":37.8, \"tF\":100.0}"}
2023-02-15T01:52:51+01:00 debug metrics/ingest.go:42 Got message {"topic": "shellyplus1-7c87ce63680c/status/temperature:100", "payload": "{\"id\": 100,\"tC\":37.6, \"tF\":99.7}"}But the data was not showing up in prometheus and a quick search ended up in this issue. The answer was in the project already: https://github.com/hikhvar/mqtt2prometheus/blob/master/hack/prometheus.yml
We need to tell prometheus to scrape the just created mqtt2prometheus. From the command line option listing https://github.com/hikhvar/mqtt2prometheus#commandline it says the default HTTP port used to expose metrics is "9641". Since not explicitly defined otherwise when executing ./mqtt2prometheus.linux_amd64 -config config.yaml this port was used.
So prometheus now needs to be configured to scrape this port "9641". Therefore edit the prometheus.yml:
sudo nano /etc/prometheus/prometheus.ymland add following: (in my case the prometheus server and the mqtt2prometheus service on port "9641" run on the same host, so I can use localhost. Otherwise tell prometheus the IP of the host running mqtt2prometheus).
- job_name: mqtt2prometheus
honor_timestamps: true
scrape_interval: 15s
scrape_timeout: 10s
metrics_path: /metrics
scheme: http
static_configs:
- targets: ['localhost:9641']Restart the prometheus server and check if it came up right:
sudo service prometheus restart
sudo service prometheus status
#exit with 'q'Note: You might need to add port TCP "9641" to firewall rules on the host running mqtt2prometheus service, so that prometheus can access the data.
Now it's time to check in browser, if prometheus get's the metrics:
http://MYIPADDRESS:9641/metricsAnd browser shows the data finally:
# TYPE heizung gauge
heizung{sensor="temperature:100",sensor_type="heizung",topic="shellyplus1-7c87ce63680c/status/temperature:100"} 34.6 1676426976363
# HELP mqtt2prometheus_connected is the mqtt2prometheus exporter connected to the broker
# TYPE mqtt2prometheus_connected gauge
mqtt2prometheus_connected 1From here, I could find in Grafana as well and render a nice graph.
I hope this solves your problem as well.
riodda
Hi, i'm reading a topic from an mqtt broker, the topic is written by ecowitt2mqtt python script and it stores this string into the topic:
I've made the config.yaml as follows:
Starting with debug options i have this console message
But the message is not sent to preometeus, as in it's log and inteface i have no trace of the message.