thingsboard / thingsboard-gateway

Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols
https://thingsboard.io/docs/iot-gateway/what-is-iot-gateway/
Apache License 2.0
1.75k stars 845 forks source link

[HELP] Dynamic keys with Default JSON Converter Type #305

Closed hamed-boumaiza-sofrecom closed 4 years ago

hamed-boumaiza-sofrecom commented 4 years ago

Hi,

Is that possible to make the key a dynamic input with the Default JSON Converter Type?

image

For exemple if i have deviceName=DEVICE-GW400

mosquitto_pub.exe -d -h localhost -p 1883 -t /central/add -m "{\"deviceName\": \"DEVICE-GW400\"}"

So if i type that command i would like that Key=DEVICE-GW400 & Value=on image

PS : I need to add/update devices each time i type that command.

imbeacon commented 4 years ago

Hi @hamed-boumaiza-protectline

Thank you for your interest in ThingsBoard IoT gateway. This is a good feature, thanks. I've implemented it, you can install the version from the develop/2.4-python branch and check it.

hamed-boumaiza-sofrecom commented 4 years ago

Hi again,

Thank you for fasty reply :) After downloading the dev/2.4-python and installing it, I typed this command

mosquitto_pub.exe -d -h test.mosquitto.org -p 1883 -t /sensor/data -m "{\"serialNumber\": \"SN001\", \"sensorType\": \"Thermometer\", \"sensorModel\": \"T1000\", \"temp\": 42, \"hum\": 58}"

And this is what i got

""2020-05-15 13:44:52" - ERROR - [json_mqtt_uplink_converter.py] - json_mqtt_uplink_converter - 77 - Error in converter, for config: {"type": "json", "deviceNameJsonExpression": "${serialNumber}", "deviceTypeJsonExpression": "${sensorType}", "timeout": 60000, "attributes": [{"type": "string", "key": "model", "value": "${sensorModel}"}, {"type": "string", "key": "${sensorModel}", "value": "on"}], "timeseries": [{"type": "double", "key": "temperature", "value": "${temp}"}, {"type": "double", "key": "humidity", "value": "${hum}"}]} and message: {'serialNumber': 'SN001', 'sensorType': 'Thermometer', 'sensorModel': 'T1000', 'temp': 42, 'hum': 58} " ""2020-05-15 13:44:52" - ERROR - [json_mqtt_uplink_converter.py] - json_mqtt_uplink_converter - 78 - argument of type 'int' is not iterable" Traceback (most recent call last): File "C:\Program Files (x86)\Python38-32\lib\site-packages\thingsboard_gateway-2.3.0.1-py3.8.egg\thingsboard_gateway\connectors\mqtt\json_mqtt_uplink_converter.py", line 66, in convert if ("${" not in value and "}" not in value) \ TypeError: argument of type 'int' is not iterable ""2020-05-15 13:44:52" - INFO - [mqtt_connector.py] - mqtt_connector - 343 - Successfully converted message from topic /sensor/data"

PS: I used the same config files.

Thanks.

imbeacon commented 4 years ago

@hamed-boumaiza-protectline

I'm also encountering this issue when test it, it is has already been fixed, just make git pull, remove old version using pip3 uninstall thingsboard-gateway and install it again.

hamed-boumaiza-sofrecom commented 4 years ago

Well that perfectly works ! Thanks @zbeacon Is it possible to use two Connectors with same topic ?

image

Both use for example /sensor/data but with different behaviour image

imbeacon commented 4 years ago

@hamed-boumaiza-protectline If you will do like in the example, the one more connector will be created. Just add one more device in the MQTT configuration file in the mapping section. Yes you can use the same topic for different devices.

hamed-boumaiza-sofrecom commented 4 years ago

Can you explain Just add one more device in the MQTT configuration file in the mapping section. ?

imbeacon commented 4 years ago

mapping section in the mqtt.json - is an array of the objects. Every object configuration is a configuration for different devices, you can just add one more object and configure it.

For example (2 different devices, with the same topics):

  "mapping": [
    {
      "topicFilter": "/sensor/data",
      "converter": {
        "type": "json",
        "deviceNameJsonExpression": "DEVICE1",
        "deviceTypeJsonExpression": "${sensorType}",
        "timeout": 60000,
        "attributes": [
          {
            "type": "string",
            "key": "model",
            "value": "${sensorModel}"
          }
        ],
        "timeseries": [
          {
            "type": "double",
            "key": "temperature",
            "value": "${temp}"
          },
          {
            "type": "double",
            "key": "humidity",
            "value": "${hum}"
          }
        ]
      }
    },
    {
      "topicFilter": "/sensor/data",
      "converter": {
        "type": "json",
        "deviceNameJsonExpression": "DEVICE2",
        "deviceTypeJsonExpression": "${sensorType}",
        "timeout": 60000,
        "attributes": [
          {
            "type": "string",
            "key": "model",
            "value": "${sensorModel}"
          }
        ],
        "timeseries": [
          {
            "type": "double",
            "key": "temperature",
            "value": "${temp}"
          },
          {
            "type": "double",
            "key": "humidity",
            "value": "${hum}"
          }
        ]
      }
    },
]
hamed-boumaiza-sofrecom commented 4 years ago

No i like it like this. I need 2 files like "mqtt1.json" & "mqtt2.json"

and its contents like this :

mqtt1.json "mapping": [ { "topicFilter": "/sensor/data", "converter": { "type": "json", "deviceNameJsonExpression": "${sensorName}", "deviceTypeJsonExpression": "${sensorType}", "timeout": 60000, "attributes": [ { "type": "string", "key": "model", "value": "${sensorModel}" } ], "timeseries": [ { "type": "double", "key": "temperature", "value": "${temp}" } ] } } ]

mqtt2.json "mapping": [ { "topicFilter": "/sensor/data", "converter": { "type": "json", "deviceNameJsonExpression": "${sensorName}", "deviceTypeJsonExpression": "${sensorType}", "timeout": 60000, "attributes": [ { "type": "string", "key": "model", "value": "${sensorModel}" } ], "timeseries": [ { "type": "double", "key": "humidity", "value": "${hum}" } ] } } ]

PS: The difference is in timeseries****

imbeacon commented 4 years ago

You can do like this, but it will be run as a different connector.

mpbr13 commented 2 years ago

Hello, I've installed tb gateway a few days ago so I'm newbie in this and I'm struggling with mapping. I'm using version 3.3.2 and I've been following this guide so I have added an entry on array in mqtt.json file which is shown below. mapping tb gateway I though that deviceNameTopicExpression was taken dynamically from {$serialNumber} but when I check in ThingsBoard this is what I get: devices tb gateway However, in last telemetry section, it shows the latests values of timeseries mapping section ({$Property} and {$Value}). So I think it is not working or I don't understand the process. Below is the telemetry sent from ESP8266. sensor tb gateway I would be so thankful if you could help me.

imbeacon commented 2 years ago

Hi @mpbr13 ,

If you need a help - please create a new issue instead of writing to the closed one, because sometimes we cannot see notification about your message.

Whenever in your case you can use the following configurations:

  1. Using deviceNameJsonExpression - it looks like you have a device name in your message, so you can just use default configuration and set "deviceNameJsonExpression":"${serialNumber}" - in this case device name will be mq135 mesa
  2. Using deviceNameTopicExpression - in order to get device name from topic you should change topicFilter to "sensor/data/+/mesa" and deviceNameTopicExpression to regular expression "(?<=sensor\/data\/)(.*?)(?=\/mesa)".

Thank you for your interest in ThingsBoard IoT gateway.

mpbr13 commented 2 years ago

Thanks a lot @zbeacon!!