[HELP] TB-Gateway doesnt push messages to Thingboard

[thaxir]

Describe the issue Create description about your issue, and your actions to solve it. In my machine there is a

1. TB-PE running port 1993
2. TB-Iot gateway running locally
3. An Mqtt broker running in docker in port 1883

In the TB-Iot Gateway i have declared the necessary info for connection to TB-PE (see the config below) In the TB-IoT Gateway i have also the default mqtt.json for handling the data I send to the mqtt broker (see the config also below) While sending to /sensor/data (mosquitto_pub -d -q 1 -h localhost -p "1883" -t "/sensor/data" -u "username" -P "password" -m '{"serialNumber":"Device A", "sensorType":"thermostat", "sensorModel":"Model A","temp":42,"hum":21}') the broker receives it , the TB-IoT Gateway reports nothing . In some tests when for example i send something wrong like (mosquitto_pub -d -q 1 -h localhost -p "1883" -t "/sensor/data" -u "username" -P "password" -m '{"serialNumber":"Device A", "sensorType":"thermostat", "model":"Model A", "temperature":42,"humidity":21}') TB-IoT gateway complains for error ([tb_utility.py] - tb_utility - 68 - No telemetry and attributes in data: {"deviceName": "Device A", "deviceType": "thermostat", "attributes": [], "telemetry": []}")

In the TB-PE i see no metrics reported from the Iot Gateway . The only metrics i can get is when i send directly to TB-PE . Configuration (Attach your configuration file) `thingsboard: host: localhost port: 1993 remoteShell: false remoteConfiguration: false statsSendPeriodInSeconds: 3600 checkConnectorsConfigurationInSeconds: 60 security: accessToken: *** qos: 1 storage: type: memory read_records_count: 100 max_records_count: 100000 connectors:

• name: MQTT Broker Connector type: mqtt configuration: mqtt.json

MQTT.json

• name: MQTT Broker Connector type: mqtt configuration: mqtt.json { "broker": { "name":"Default Local Broker", "host":"0.0.0.0", "port":1883, "clientId": "ThingsBoard_gateway", "security": { "type": "basic", "username": "username", "password": "password" } }, "mapping": [ { "topicFilter": "/sensor/data", "converter": { "type": "json", "deviceNameJsonExpression": "${serialNumber}", "deviceTypeJsonExpression": "${sensorType}", "timeout": 60000, "attributes": [ { "type": "string", "key": "model", "value": "${sensorModel}" } ], "timeseries": [ { "type": "double", "key": "temperature", "value": "${temp}" }, { "type": "double", "key": "humidity", "value": "${hum}" } ] } }, { "topicFilter": "/sensor/+/data", "converter": { "type": "json", "deviceNameTopicExpression": "(?<=sensor\/)(.?)(?=\/data)", "deviceTypeTopicExpression": "Thermometer", "timeout": 60000, "attributes": [ { "type": "string", "key": "model", "value": "${sensorModel}" } ], "timeseries": [ { "type": "double", "key": "temperature", "value": "${temp}" }, { "type": "double", "key": "humidity", "value": "${hum}" } ] } }, { "topicFilter": "/custom/sensors/+", "converter": { "type": "custom", "extension": "CustomMqttUplinkConverter", "extension-config": { "temperatureBytes": 2, "humidityBytes": 2, "batteryLevelBytes": 1 } } } ], "connectRequests": [ { "topicFilter": "sensor/connect", "deviceNameJsonExpression": "${SerialNumber}" }, { "topicFilter": "sensor/+/connect", "deviceNameTopicExpression": "(?<=sensor\/)(.?)(?=\/connect)" } ], "disconnectRequests": [ { "topicFilter": "sensor/disconnect", "deviceNameJsonExpression": "${SerialNumber}" }, { "topicFilter": "sensor/+/disconnect", "deviceNameTopicExpression": "(?<=sensor\/)(.?)(?=\/disconnect)" } ], "attributeRequests": [ { "retain": false, "topicFilter": "v1/devices/me/attributes/request", "deviceNameTopicExpression": "${SerialNumber}", "attributeNameJsonExpression": "${sensorModel}" } ], "attributeUpdates": [ { "retain": true, "deviceNameFilter": "SmartMeter.", "attributeFilter": "uploadFrequency", "topicExpression": "sensor/${deviceName}/${attributeKey}", "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}" } ], "serverSideRpc": [ { "deviceNameFilter": ".", "methodFilter": "echo", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}", "responseTimeout": 10000, "valueExpression": "${params}" }, { "deviceNameFilter": ".", "methodFilter": "no-reply", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "valueExpression": "${params}" } ] }

Error traceback (If it was raised):

""2021-10-29 20:01:03" - ERROR - [tb_utility.py] - tb_utility - 68 - No telemetry and attributes in data: {"deviceName": "Device A", "deviceType": "thermostat", "attributes": [], "telemetry": []}"

Versions (please complete the following information):

• OS: Ubuntu 18.04
• Thingsboard IoT Gateway version latest
• Python version Python 2.7.17 and Python 3.6.9

[imbeacon]

Hi @Kernelalive ,

Thank you for interest in ThingsBoard IoT gateway, please set logging level in file logs.conf from INFO to DEBUG, run the gateway, send data with the command mosquitto_pub -d -q 1 -h localhost -p "1883" -t "/sensor/data" -u "username" -P "password" -m '{"serialNumber":"Device A", "sensorType":"thermostat", "sensorModel":"Model A","temp":42,"hum":21}', stop the gateway and send logs to us. We need the following files: service.log, connector.log, converter.log, tb_connection.log

[thaxir]

Hello @zbeacon , thank you for you reply. So following your instructions i report to you the logs i have for the command you gave me Service logs service.log

*Connector Log connector.log Converter Logs converter.log Connection Logs tb_connection.log

Just to add here that my purpose is to connect multiple devices on the tb-gateway and report that to TB-PE.

Thank you in advance

[imbeacon]

Hi @Kernelalive ,

Please try to use the latest version (2.9 was released today) The issue can be in RemoteLogging feature (If it was set to DEBUG level - the gateway in release 2.8 was working wrong).

[thaxir]

Hello @zbeacon , i have redeployed the newest thingsboard-gateway. For some reason the logs are blank . The only logs i have is the following. I get this from journactl command.

Do you have any idea why this happening?

Νοε 01 16:56:49 kernelalive-VirtualBox systemd[1]: Stopping ThingsBoard Gateway... Νοε 01 16:56:49 kernelalive-VirtualBox systemd[1]: Stopped ThingsBoard Gateway. Νοε 01 16:56:49 kernelalive-VirtualBox systemd[1]: Started ThingsBoard Gateway. Νοε 01 16:56:51 kernelalive-VirtualBox python3[8390]: [STREAM ONLY] 2021-11-01 16:56:51,929 - WARNING - [tb_gateway_service.py] - tb_gateway_service - 128 - Remote shell is enabled. Please be carefully with this feature. Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: [STREAM ONLY] 2021-11-01 16:56:52,306 - ERROR - [tb_gateway_service.py] - tb_gateway_service - 334 - Error on loading connector: Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: [STREAM ONLY] 2021-11-01 16:56:52,306 - ERROR - [tb_gateway_service.py] - tb_gateway_service - 335 - Extra data: line 11 column 4 - line 132 column 1 (char 220 - 3346) Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: Traceback (most recent call last): Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: File "/usr/lib/python3/dist-packages/thingsboard_gateway/gateway/tb_gateway_service.py", line 324, in _load_connectors Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: connector_conf = load(conf_file) Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: File "/usr/lib/python3/dist-packages/simplejson/init.py", line 461, in load Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: use_decimal=use_decimal, **kw) Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: File "/usr/lib/python3/dist-packages/simplejson/init.py", line 518, in loads Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: return _default_decoder.decode(s) Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 373, in decode Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: raise JSONDecodeError("Extra data", s, end, len(s)) Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: simplejson.errors.JSONDecodeError: Extra data: line 11 column 4 - line 132 column 1 (char 220 - 3346) Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: [STREAM ONLY] 2021-11-01 16:56:52,585 - INFO - [tb_gateway_service.py] - tb_gateway_service - 290 - Remote logging has being updated. Current logging level is: DEBUG Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: [STREAM ONLY] 2021-11-01 16:56:52,585 - DEBUG - [tb_gateway_service.py] - tb_gateway_service - 293 - Shared attributes received (RemoteLoggingLevel). Νοε 01 16:56:52 kernelalive-VirtualBox python3[8390]: [STREAM ONLY] 2021-11-01 16:56:52,585 - DEBUG - [tb_gateway_service.py] - tb_gateway_service - 296 - Client attributes received (current_configuration).

[thaxir]

Update I fixed the problems with the logs . I think it was due to a discrepancy with the previous install of tb-gw 2.7 . Now i see the the messages sent by mqtt publisher . So i send mosquitto_pub -d -q 1 -h localhost -p "1883" -t "/sensor/data" -u "username" -P "password" -m '{"serialNumber":"Device A", "sensorType":"thermostat", "sensorModel":"Model A","temp":42,"hum":21}' and i can see them on the TB-Gw logs as you will see below. BTW ss this some bug of 2.7 version? Now My question now is : how to see / move these data to TB-PE so to handle them
And here are the logs: service.log connector.log

[imbeacon]

You can see the data on your instance of ThingsBoard - device with name "Device A" should contain attributes and telemetry on the same named tabs just click on the device in device group and open attributes tab.

[thaxir]

Hi @zbeacon , you are right ! thank you again for your support . Just another question . Does TB listens to specific topics? So can i change/program the mapping that TB-GW does to the corresponding topics that TB listens to ?