[BUG] Device type set to default after removing the device from the list then republished via gateway

[chenybing]

Describe the bug

When the gateway first publishes a device status with the deviceType set to a value (in this case it was set to the string 'server', nothing fancy or dynamic), it shows up properly in the list of devices as device type "server". Then, remove that device from the list of devices, have the gateway publish that same message again, except the device that shows up on the list is showing as device type "default".

It shows the correct device type whenever the device status first gets published to Thingsboard. Deletion, then subsequent republish, causes the device type to be set as "default". Another behavior is that the device doesn't show up on the list at all for the second publishing.

Background

Our setup is such that we have a server that publishes status to a RabbitMQ Broker. We wrote a custom connector that subscribes to the RabbitMQ Broker and receives the message in JSON and publishes to the Thingsboard MQTT Broker.

Steps to reproduce

1. The Thingsboard Gateway service is started, we publish the following message to it via our RabbitMQ connector:

JSON Message: { "deviceName": "Test Script Device", "deviceType": "test script", "overallHealth": "NORMAL", "state": "OPERATIONAL", "temp_s1": "888.88", "temp_front_card": "888.88", "temp_rear_card": "888.88", "hum_s1": "888.88", "hum_front_card": "888.88", "hum_rear_card": "888.88", "power": "888.88" }

Python script:

#!/usr/bin/python3

import pika
import json
import sys

connection = pika.BlockingConnection(
    pika.ConnectionParameters(host='localhost'))
channel = connection.channel()

channel.exchange_declare(exchange='topic_logs', exchange_type='topic', durable=True)

routing_key='site.A.servers.attributes'

f = open('test.json',)

data = json.load(f)
channel.basic_publish(exchange='topic_logs', routing_key=routing_key, body=json.dumps(data))

print(" [x] Sent %r:%r" % (routing_key, data))
connection.close()

2. Device "Test Script Device" and device type "test script" shows up in the list of Devices on Thingsboard.
3. Delete device "Test Script Device".
4. Re-send the same message as above via RabbitMQ.
5. Device either no longer shows up on the list of device, OR the device shows up, but the device type is now set to "default" instead of "test script".

Error traceback (If available):

Not so much an error traceback (I didn't see any), but a verification that our connector is working correctly on our end (from connector.log):

""2020-10-20 16:12:33" - INFO - [rabbitmq_connector.py] - rabbitmq_connector - 122 - Routing Key: 'site.A.servers.attributes'"
""2020-10-20 16:12:33" - INFO - [rabbitmq_connector.py] - rabbitmq_connector - 127 - topic handlers are: ['site.*.servers.attributes']"
""2020-10-20 16:12:33" - INFO - [rabbitmq_connector.py] - rabbitmq_connector - 144 - CONVERTED CONTENT: {'deviceName': 'Test Script Device', 'deviceType': 'test script', 'attributes': [{'health': 'NORMAL'}, {'state': 'OPERATIONAL'}], 'telemetry': {'ts': 1603224753843, 'values': {'power usage': '888.88', 'temperature sensor #1': '888.88', 'front card temperature sensor': '888.88', 'rear card temperature sensor': '888.88', 'humidity sensor #1': '888.88', 'front card humidity sensor': '888.88', 'rear card humidity sensor': '888.88'}}}"

Versions (please complete the following information):

• OS: Ubuntu 20.04 LTS 64 bit (running as an image in VMWare ESXi)
• Thingsboard version: 3.1.1 PE
• Thingsboard Gateway version: 2.5.1 (latest)
• RabbitMQ 3.8.2
• Python 3.8 (default)

[chenybing]

I also want to mention that when I reinstall thingsboard-gateway, and try to send data with the same device name and device type, it displays correctly on Thingsboard. Delete the device from Thingsboard, rinse, repeat, the problem resurfaces.

[imbeacon]

Hi @chenybing ,

Thank you for your feedback, we found the cause of the issue and gonna fix it in ThingsBoard. This issue was appeared because the gateway didn't know that the device was removed on ThingsBoard, so we will add one more topic to the Gateway MQTT API and processing for these messages.

[chenybing]

Cheers @zbeacon . Do you have an approximation of which future release the fix will be applied?

[imbeacon]

Hi @chenybing ,

It is not so regular behavior for the gateway to keep which devices were removed from the ThingsBoard and we didn't meet requirements like this earlier. But we have developed the functionality like you described. Unfortunately, I don't have concrete date when it will be available, I hope it will be added into ThingsBoard version 3.2 and also I will publish the new version of the gateway when this functionality will be available.

As a workaround you can remove file "connected_devices.json" when gateway stops and after start it will create the device with correct device type.

[chenybing]

Thanks for the workaround @zbeacon .

[chenybing]

Just a thought: Is there a MQTT topic I can subscribe to on Thingsboard that can notify me when a device has been added/removed?

[imbeacon]

We have the same thought, and implemented it, to send data to the gateway about the removed devices, but it is also will be available only in ThingsBoard 3.2.

[hazarmutgan]

@zbeacon is there any update on this since the ThingsBoard 3.2 is now available?