# in MQTT_TOPIC not working.

[simonebenati]

Describe the bug A clear and concise description of what the bug is

the client is subscribing to # but the exporter doesn't have any metrics

To Reproduce Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior mqtt-exporter: image: kpetrem/mqtt-exporter ports:

• 9000:9000 environment:
  • MQTT_ADDRESS=url.info.com
  • MQTT_PORT=1883
  • MQTT_USERNAME=user
  • MQTT_PASSWORD=pass
  • LOG_LEVEL=DEBUG
  • KEEP_FULL_TOPIC=True

Screenshots If applicable, add screenshots to help explain your problem.

Additional context When i then query my /metrics at my address there is no metrics shown whatsoever.

DEBUG:mqtt-exporter:Sending CONNECT (u1, p1, wr0, wq0, wf0, c1, k60) client_id=b''
DEBUG:mqtt-exporter:Received CONNACK (0, 0)
INFO:mqtt-exporter:subscribing to "#"
DEBUG:mqtt-exporter:Sending SUBSCRIBE (d0, m1) [(b'#', 0)]
DEBUG:mqtt-exporter:Received SUBACK

[simonebenati]

Actually I found that if there's strings it will throw this error:

DEBUG:mqtt-exporter:New message from MQTT: xxx/UPD/11/U/FILE - b'{"test" : "yulj"}' DEBUG:mqtt-exporter:Failed to convert test: could not convert string to float: 'YULJ'

And therefore if you put # it will incorporate some strings and therefore consistently fail.

[kpetremann]

hello @simonebenati,

Ok so good news, the exporter is successfully connected to your MQTT server.

It is frequent to have events in MQTT that cannot be converted to a Prometheus metrics. The value must be a number or converted to integer. "YUJL" cannot be converted.

Example of supported events:

• topic '/', payload '{"temperature":26.24,"humidity":45.37}'
• topic '//sensor/temperature' '20.00'

Could you please share the events you have in MQTT that you want to be in Prometheus?

[simonebenati]

Hello @kpetremann , thank you for your help but I wanted to write to a topic the name of a project i.e “SHIRT” that is currently being taken by machine and elaborated. Next to this project name i have a counter for how many shirts my machine is producing

[simonebenati]

The thing is, regardless if I’m connected or not if i put # as topic i won’t receive anything not even the topic with a number as a value

[kpetremann]

I don't understand your meaning. The log you provided (DEBUG:mqtt-exporter:New message from MQTT: xxx/UPD/11/U/FILE - b'{"test" : "yulj"}') shows that the exporter is getting the message, but cannot be parsed.

It shows a MQTT event that cannot be processed: xxx/UPD/11/U/FILE - b'{"test" : "yulj"}' The value must contain a numeric, not a string, because Prometheus is expecting a string as value.

Please provide the MQTT event you have and the Prometheus metric you are expecting, so I can better understand the issue.

[kpetremann]

The topics must follow the formats which are supported. It is documented in the README

For instance you could have:

topic: app_name/machine1/SHIRT/counter
payload: 123

[kpetremann]

And therefore if you put # it will incorporate some strings and therefore consistently fail.

You can subscribe to everything using #. The failure you see is just for message that cannot be parsed. They are just skipped.

[simonebenati]

Sorry there are two issues at play let me clarify that.

issue n1: When i set topic env variable to # (or default) and i have mixed topics some with numbers as values and some as strings. I receive no metrics, none of them.

issue n2: I would like the exporter to be able to read a string from the topic, is it possible? Or if it reads a string manipulate the metric so that it throws it in as a label? That i can query then in prometheus and do a group_left for instance with a counter (this is me guessing, I’m not an expert and maybe just fantasizing)

topic: app_name/machine1/SHIRT/type payload: ”blue”

[kpetremann]

Thanks for the clarification.

issue 1

Please provide some examples (topics and payloads).

issue 2

This is not supported. Payload must be a number, and the only label produced is "topic".

If you successfully get all the info you need in the topic, and a number in the payload (like the counter), you should be able to create new labels from the topic label using a Prometheus recording rule.

An alternative would be to try a more customizable exporter, like this one: https://github.com/hikhvar/mqtt2prometheus

[simonebenati]

Hello @kpetremann , I could export the mqtt output this morning only. mqtt.txt

Here you can see the mqtt topics that if I subscribe with the default # setting it won't load any of these.

[kpetremann]

hello @simonebenati,

This is weird, I have just tested with interface test mode (which can be found in PR #80), and it successfully creates the metric:

❯ python ./exporter.py --test
# Test conversion from a topic/payload parsing:

topic: customer/UPD/11/U/MAXPOS
payload: 12

parsed to: customer_upd_11_u {'MAXPOS': 12}
INFO:mqtt-exporter:creating prometheus metric: PromMetricId(name='mqtt_MAXPOS', labels=())

# Metrics:
# HELP mqtt_MAXPOS metric generated from MQTT message.
# TYPE mqtt_MAXPOS gauge
mqtt_MAXPOS{topic="customer_upd_11_u"} 12.0

Which MQTT server are you using? Mosquitto?

[kpetremann]

I did another test.

I pushed the topic and payload to my MQTT server:

mosquitto_pub -t customer/UPD/11/U/MAXPOS -m 45 -u user -P password

And I do see the new metric:

$ curl -s 127.0.0.1:9000/metrics | grep -v "#" | grep -i maxpos
mqtt_MAXPOS{topic="customer_upd"} 45.0

I used this config:

      - MQTT_ADDRESS=mqtt
      - MQTT_USERNAME=user
      - MQTT_PASSWORD=password
      - LOG_LEVEL=DEBUG
      - KEEP_FULL_TOPIC=true

[simonebenati]

These actually works because the payload is a number and are singlets. The mqtt values payloads in my file are defaults that retry even when the machine is turned off. If you manage to push them to mosquitto all (all 100+) at the same time they don't work. (atleast in my case)

[kpetremann]

Yes payload must be a number or JSON with nested numbers, otherwise the message is ignored and the exporter will just proceed to the next messages in MQTT. There is no plan to support string as payload.

I have some questions:

• I don't get the "they are singlets", please provide more details about this.

• Please also provide the output of /metrics of the exporter.

• Are you saying that one invalid message is making the exporter failing to expose the rest of the valid messages as well?

[kpetremann]

I am a bit confused. I have pushed all the messages from your mqtt.txt file, and the exporter is successfully exposing these metrics:

mqtt_ALM{topic="customer_upd"} 0.0
mqtt_SPEED{topic="customer_upd"} 62.0
mqtt_AUTO{topic="customer_upd"} 1.0
mqtt_STAT{topic="customer_upd"} 3.0
mqtt_PW{topic="customer_upd"} 1.0
mqtt_MSG{topic="customer_upd"} 0.0
mqtt_MAXPOS{topic="customer_upd"} 8.0
mqtt_POS{topic="customer_upd"} 7.0
mqtt_COUNT{topic="customer_upd"} 178.0

Note: I have pushed them in a for loop, so technically it is done one by one. But even after a restart of mqtt-exporter, the metrics are still exposed again, as it re-reads and processes all messages in MQTT.

[simonebenati]

I am confused aswell then. In my system when I subscribe to topic # they don't get taken in (not even those with a number as a payload).. How can I help you debug?

[kpetremann]

maybe by sharing:

• the entire configuration and version of mosquitto
• the config and version (or the commit HEAD you use) of mqtt-exporter
• share all the logs you have of mqtt-exporter (if there are others logs than what you already shared)
• the output of /metrics of mqtt-exporter

I hope I will be able to reproduce your issue.

[kpetremann]

any update?

[kpetremann]

I am closing the issue. Feel free to re-open it you still have the issue (if so, please provide the requested info). Thanks.