shirou / mqforward

MQTT to influxdb forwarder
MIT License
75 stars 24 forks source link

mqtt from The Things Network can not be decoded #9

Open mausquirk opened 7 years ago

mausquirk commented 7 years ago

The Things Network (TTN) serves the following message to mqforward.

{
  "analog_in_4": 0.94,
  "barometric_pressure_1": 0.5,
  "relative_humidity_3": 127.5,
  "temperature_2": -1
}

mqforward seems to have trouble to decode this JSON messages:

INFO[0000] Broker URI: tcp://eu.thethings.network:1883  
INFO[0000] connecting...                                
INFO[0000] influxdb host: https://iot.bla.bla:8086   
INFO[0000] mqtt connected                               
INFO[0000] subscribed: map[miromicrotest/devices/#:0]   
INFO[0000] influxdb connected.                          
INFO[0258] topic:miromicrotest/devices/multisensorhub1/up 
INFO[0258] topic:miromicrotest/devices/multisensorhub1/up/barometric_pressure_1 
INFO[0258] topic:miromicrotest/devices/multisensorhub1/up/temperature_2 
INFO[0258] topic:miromicrotest/devices/multisensorhub1/up/relative_humidity_3 
INFO[0258] topic:miromicrotest/devices/multisensorhub1/up/analog_in_4 
WARN[0259] json: cannot unmarshal number into Go value of type map[string]interface {} 
WARN[0259] json: cannot unmarshal number into Go value of type map[string]interface {} 
WARN[0259] json: cannot unmarshal number into Go value of type map[string]interface {} 
WARN[0260] json: cannot unmarshal number into Go value of type map[string]interface {} 

The measurement multisensorhub1.up exists in the target influxdb.

Do you have any experience with TTN? Might this be a configuration issue or shall I dive into the GO-code?

thanks.

shirou commented 7 years ago

I have not use https://www.thethingsnetwork.org/. But, map[string]interface {} should unmarshal about your sample JSON. Something wrong payload?

DuaneWayne commented 6 years ago

I'm having the same issue with some iot sensors sending mqtt messages. The messages seem to be read fine, but I see errors when it starts to send the message to influx, and no data appears in the database.

Any assistance and/or pointers about where to get started much appreciated. The github link to the publisher code is at the bottom.

grant@IoT-Srv:~/go/src/github.com/shirou/mqforward$ mqforward run INFO[0000] Broker URI: tcp://10.0.0.118:1883
INFO[0000] connecting...
INFO[0000] influxdb host: http://127.0.0.1:8086
INFO[0000] mqtt connected
INFO[0000] subscribed: map[sensor/#:0]
INFO[0000] influxdb connected.
INFO[0000] topic:sensor/1/temperature/degreeCelsius
INFO[0000] topic:sensor/1/temperature/degreeFahrenheit
INFO[0000] topic:sensor/1/humidity/percentRelative
INFO[0000] topic:sensor/1/barometer/hectoPascal
INFO[0000] topic:sensor/1/barometer/inchHg
INFO[0000] topic:sensor/MasterBedroom/temperature/degreeCelsius DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
INFO[0000] topic:sensor/MasterBedroom/temperature/degreeFahrenheit INFO[0000] topic:sensor/MasterBedroom/humidity/percentRelative INFO[0000] topic:sensor/MasterBedroom/barometer/hectoPascal INFO[0000] topic:sensor/MasterBedroom/barometer/inchHg
INFO[0000] topic:sensor/DaddyUpstairsOffice/temperature/degreeCelsius DEBU[0000] add: 1/temperature/degreeCelsius
DEBU[0000] add: 1/temperature/degreeFahrenheit
DEBU[0000] add: 1/humidity/percentRelative
DEBU[0000] add: 1/barometer/hectoPascal
DEBU[0000] add: 1/barometer/inchHg
DEBU[0000] add: MasterBedroom/temperature/degreeCelsius DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] add: MasterBedroom/temperature/degreeFahrenheit DEBU[0000] add: MasterBedroom/humidity/percentRelative
DEBU[0000] add: MasterBedroom/barometer/hectoPascal
DEBU[0000] add: MasterBedroom/barometer/inchHg
DEBU[0000] add: DaddyUpstairsOffice/temperature/degreeCelsius INFO[0000] topic:sensor/DaddyUpstairsOffice/temperature/degreeFahrenheit INFO[0000] topic:sensor/DaddyUpstairsOffice/humidity/percentRelative INFO[0000] topic:sensor/DaddyUpstairsOffice/barometer/hectoPascal INFO[0000] topic:sensor/DaddyUpstairsOffice/barometer/inchHg INFO[0000] topic:sensor/temperature
INFO[0000] topic:sensor/humidity
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
INFO[0000] topic:sensor/FrontPatio/temperature
INFO[0000] topic:sensor/FrontPatio/temperature/degreeCelsius INFO[0000] topic:sensor/FrontPatio/humidity
INFO[0000] topic:sensor/FrontPatio/humidity/percentRelative INFO[0000] topic:sensor/BackPatio/temperature/degreeCelsius DEBU[0000] add: DaddyUpstairsOffice/temperature/degreeFahrenheit DEBU[0000] add: DaddyUpstairsOffice/humidity/percentRelative DEBU[0000] add: DaddyUpstairsOffice/barometer/hectoPascal DEBU[0000] add: DaddyUpstairsOffice/barometer/inchHg
DEBU[0000] add: temperature
DEBU[0000] add: humidity
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] add: FrontPatio/temperature
DEBU[0000] add: FrontPatio/temperature/degreeCelsius
DEBU[0000] add: FrontPatio/humidity
DEBU[0000] add: FrontPatio/humidity/percentRelative
DEBU[0000] add: BackPatio/temperature/degreeCelsius
INFO[0000] topic:sensor/BackPatio/temperature/degreeFahrenheit INFO[0000] topic:sensor/BackPatio/humidity/percentRelative INFO[0000] topic:sensor/BackPatio/barometer/hectoPascal INFO[0000] topic:sensor/BackPatio/barometer/inchHg
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] msg comes from mqtt
DEBU[0000] add: BackPatio/temperature/degreeFahrenheit
DEBU[0000] add: BackPatio/humidity/percentRelative
DEBU[0000] add: BackPatio/barometer/hectoPascal
DEBU[0000] add: BackPatio/barometer/inchHg
DEBU[0001] send to influxdb: size=8
WARN[0001] json: cannot unmarshal number into Go value of type map[string]interface {} WARN[0001] json: cannot unmarshal number into Go value of type map[string]interface {} WARN[0001] json: cannot unmarshal number into Go value of type map[string]interface {} WARN[0001] json: cannot unmarshal number into Go value of type map[string]interface {} DEBU[0002] send to influxdb: size=4
WARN[0002] json: cannot unmarshal number into Go value of type map[string]interface {} WARN[0002] json: cannot unmarshal number into Go value of type map[string]interface {} WARN[0002] json: cannot unmarshal number into Go value of type map[string]interface {} WARN[0002] json: cannot unmarshal number into Go value of type map[string]interface {}

the data in mqtt looks like this:

mosquitto_sub -h 10.0.0.118 -p 1883 -v -t sensor/#

sensor/1/temperature/degreeCelsius 24.80 sensor/1/temperature/degreeFahrenheit 76.64 sensor/1/humidity/percentRelative 26.15 sensor/1/barometer/hectoPascal 1048.40 sensor/1/barometer/inchHg 30.96 sensor/MasterBedroom/temperature/degreeCelsius 24.36 sensor/MasterBedroom/temperature/degreeFahrenheit 75.85 sensor/MasterBedroom/humidity/percentRelative 22.78 sensor/MasterBedroom/barometer/hectoPascal 1048.76 sensor/MasterBedroom/barometer/inchHg 30.97 sensor/DaddyUpstairsOffice/temperature/degreeCelsius 22.28 sensor/DaddyUpstairsOffice/temperature/degreeFahrenheit 72.10 sensor/DaddyUpstairsOffice/humidity/percentRelative 27.24 sensor/DaddyUpstairsOffice/barometer/hectoPascal 1048.19 sensor/DaddyUpstairsOffice/barometer/inchHg 30.95 sensor/temperature 25.50 sensor/humidity 32.90 sensor/FrontPatio/temperature 23.20 sensor/FrontPatio/temperature/degreeCelsius 17.70 sensor/FrontPatio/humidity 34.20 sensor/FrontPatio/humidity/percentRelative 51.50 sensor/BackPatio/temperature/degreeCelsius 5.49 sensor/BackPatio/temperature/degreeFahrenheit 41.88 sensor/BackPatio/humidity/percentRelative 37.55 sensor/BackPatio/barometer/hectoPascal 1048.41 sensor/BackPatio/barometer/inchHg 30.96

and the sender is running this software:

https://gist.github.com/mtl010957/9ee85fb404f65e15c440b08c659c0419

zcattacz commented 2 years ago

I had a device that would send trailing \x0 (c null char) in json. took some time to bug the vendor to fix. It was figured out in a hexdump.