thingsboard / thingsboard-gateway

Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols
https://thingsboard.io/docs/iot-gateway/what-is-iot-gateway/
Apache License 2.0
1.72k stars 829 forks source link

Data from MQTT Broker Connector connector is invalid #425

Closed salemsw closed 3 years ago

salemsw commented 3 years ago

I installed ThingsBoard-tb-postgres, ThingsBoard-tb-gateway, and mosquitto using Docker, gateway receives the message with this error in service.log : ERROR - [tb_gateway_service.py] - tb_gateway_service - 322 - Data from MQTT Broker Connector connector is invalid." ERROR - [tb_utility.py] - tb_utility - 63 - No telemetry and attributes in data: {"deviceName": "Store", "deviceType": "Store", "attributes": [], "telemetry": []}"

and this error from converter.log : ""2020-10-29 16:12:02" - ERROR - [json_mqtt_uplink_converter.py] - json_mqtt_uplink_converter - 77 - Error in converter, for config: {"type": "json", "deviceNameTopicExpression": "(?<=ajman/salem/)(.?)(?=/data)", "deviceTypeTopicExpression": "(?<=ajman/salem/)(.*?)(?=/data)", "timeout": 60000, "attributes": [{"type": "string", "key": "model", "value": "${sensorModel}"}], "timeseries": [{"type": "double", "key": "temperature", "value": "${temp}"}]} and message: { “sensorModel”: “T1000”, “temp”: 11} " ""2020-10-29 16:12:02" - ERROR - [json_mqtt_uplink_converter.py] - json_mqtt_uplink_converter - 78 - Expecting property name enclosed in double quotes: line 1 column 3 (char 2)" Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/thingsboard_gateway-2.5.2-py3.7.egg/thingsboard_gateway/connectors/mqtt/json_mqtt_uplink_converter.py", line 62, in convert value = TBUtility.get_value(datatype_config["value"], data, datatype_config["type"], expression_instead_none=True) File "/usr/local/lib/python3.7/site-packages/thingsboard_gateway-2.5.2-py3.7.egg/thingsboard_gateway/tb_utility/tb_utility.py", line 120, in get_value body = loads(body) File "/usr/local/lib/python3.7/site-packages/simplejson-3.17.2-py3.7-linux-x86_64.egg/simplejson/init.py", line 525, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.7/site-packages/simplejson-3.17.2-py3.7-linux-x86_64.egg/simplejson/decoder.py", line 370, in decode obj, end = self.raw_decode(s) File "/usr/local/lib/python3.7/site-packages/simplejson-3.17.2-py3.7-linux-x86_64.egg/simplejson/decoder.py", line 400, in raw_decode return self.scan_once(s, idx=_w(s, idx).end()) simplejson.errors.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 3 (char 2)

Configuration thingsboard: host: 192.168.1.227 port: 1884 remoteShell: false remoteConfiguration: false security: accessToken: ***** qos: 1 storage: type: memory read_records_count: 100 max_records_count: 100000

type: file

data_folder_path: ./data/

max_file_count: 10

max_read_records_count: 10

max_records_per_file: 10000

connectors:

name: MQTT Broker Connector
type: mqtt
configuration: mqtt.json

Connector name : MQTT Connector

Versions :

salemsw commented 3 years ago

and this is mqtt.json: { "broker": { "name":"Default Local Broker", "host":"192.168.1.227", "port":1883, "clientId": "ThingsBoard_gateway", "security": { "type": "basic", "username": "spectrum", "password": "Dw7Kx5Vj3" } }, "mapping": [ { "topicFilter": "/ajman/salem/+/data", "converter": { "type": "json", "deviceNameTopicExpression": "(?<=ajman\/salem\/)(.*?)(?=\/data)", "deviceTypeTopicExpression": "(?<=ajman/salem\/)(.*?)(?=\/data)", "timeout": 60000, "attributes": [ { "type": "string", "key": "model", "value": "${sensorModel}" } ], "timeseries": [ { { "type": "double", "key": "temperature", "value": "${temp}" } ] } } ], "connectRequests": [ { "topicFilter": "ajman/salem/+/data", "deviceNameJsonExpression": "(?<=ajman/salem/)(.*?)(?=/data)" } ], "serverSideRpc": [], "disconnectRequests": [], "attributeRequests": [], "attributeUpdates": [] }

imbeacon commented 3 years ago

Hi @salemsw ,

Thank you for your feedback. This issue caused by json decoder, it cannot parse incoming json.

salemsw commented 3 years ago

Hi @zbeacon ,

Thank you for your reply, so what I understand I have to wait for next gateway update to solve this issue.

imbeacon commented 3 years ago

No, it looks like some issue with payload in the incoming message. Could you attach the payload that you sent?

salemsw commented 3 years ago

This is the template: { “sensorModel”: “T1000”, “temp”: $$VALUE:}

and this is what it shows sent: { “sensorModel”: “T1000”, “temp”: 15.6}

imbeacon commented 3 years ago

There are wrong symbols “ and ” are not equal to "

salemsw commented 3 years ago

thank you very much, I was editing this payload remotely from my mac, and really not easy to discover that, thanks a lot. now it works

deepthi30192 commented 3 years ago

Hi @salemsw how do we give the payload? Where should we specify it