[HELP] MQTT connector failure

[mirtaheri]

Problem After an attempt to upgrade the gateway, an issue occured which remained even after downgrading to the previous version and even re-installing server. Note that all the configurations are exactly the ones used in the working version (a part from device token of course). Problem is that initially the values are catched as keys, as the following image:

Then after re-installing GW and Server, sensor's messages don't get parsed; considering following example, temp variable is not read from sensor's message

   "timeseries": [
          {
            "type": "double",
            "key": "temperature",
            "value": "${temp}"
          }

to testing that if we put constant value ("value": "0") it works, meaning that the keys are correct.

Configuration This is the configuration file for the Gateway:

thingsboard:
  host: 130.xxx.xxx.xxx
  port: 1883
  remoteConfiguration: false
  security:
    accessToken: XXXXXXXX 
storage:
  type: memory
  read_records_count: 100
  max_records_count: 100000

connectors:
  -
    name: MQTT WS Connector
    type: mqtt
    configuration: weatherstation.json

  -
    name: MQTT Forecast Connector
    type: mqtt
    configuration: forecast.json

Connector name : We use MQTT Connector:

{
  "broker": {
    "name": "WeatherStationBroker",
    "host": "127.0.0.1",
    "port": 1883,
    "security": {
      "type": "anonymous"
    }
  },
  "mapping": [
    {
      "topicFilter": "WeatherSensors/loop",
      "converter": {
        "type": "json",
    "deviceNameJsonExpression": "WeatherStationSensor",
        "deviceTypeJsonExpression": "Sensors",
        "timeout": 60000,
        "timeseries": [
          {
            "type": "double",
            "key": "outTemp",
            "value": "${outTemp}"
          },
          {
            "type": "double",
            "key": "highOutTemp",
            "value": "${highOutTemp}"
          },
          {
            "type": "double",
            "key": "lowOutTemp",
            "value": "${lowOutTemp}"
          },
          {
            "type": "double",
            "key": "rain",
            "value": "${rain}"
          },
          {
            "type": "double",
            "key": "rainRate",
            "value": "${rainRate}"
          },
          {
            "type": "double",
            "key": "barometer",
            "value": "${barometer}"
          },
          {
            "type": "double",
            "key": "radiation",
            "value": "${radiation}"
          },
          {
            "type": "double",
            "key": "inTemp",
            "value": "${inTemp}"
          },
          {
            "type": "double",
            "key": "inHumidity",
            "value": "${inHumidity}"
          }
        ]
      }
    }
  ],
  "connectRequests": [],
  "disconnectRequests": [],
  "attributeUpdates": [],
  "serverSideRpc": [],
  "attributeRequests": []
}

Error traceback (If it was raised):

Logs don't show any problem apparently:

""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected mapping handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted serverSideRpc handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected serverSideRpc handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted connectRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected connectRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted disconnectRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected disconnectRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted attributeRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected attributeRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted attributeUpdates handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected attributeUpdates handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted mapping handlers: 1"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected mapping handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted serverSideRpc handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected serverSideRpc handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted connectRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected connectRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted disconnectRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected disconnectRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted attributeRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected attributeRequests handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted attributeUpdates handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 227 - MQTT Davis Connector connected to 127.0.0.1:1883 - successfully."
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected attributeUpdates handlers: 0"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 227 - MQTT Forecast Connector connected to localhost:1883 - successfully."
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 267 - Connector "MQTT WS Connector" subscribe to WeatherSensors/loop"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 267 - Connector "MQTT Forecast Connector" subscribe to SolarForecastTopic"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 315 - "MQTT WS Connector" subscription success to topic WeatherSensors/loop, subscription message id = 1"
""2020-07-26 23:10:08" - INFO - [mqtt_connector.py] - mqtt_connector - 315 - "MQTT Forecast Connector" subscription success to topic SolarForecastTopic, subscription message id = 1"
'

Versions (please complete the following information):

• OS: [Description: Ubuntu Groovy Gorilla (development branch) Release: 20.10]
• Thingsboard IoT Gateway version [2.2.4]
• Python version[e.g. 3.8]

[imbeacon]

Hi @mirtaheri ,

Thank for your interest in the ThingsBoard IoT gateway.

Please attach the device message.

[mirtaheri]

Thanks for you prompt reply. Here it is the message: {"ts": "1595839980000", "interval": "60.0", "rxCheckPercent": "100.00", "outTemp": "27.0", "highOutTemp": "80.6", "lowOutTemp": "80.6", "rain": "0.00", "rainRate": "0.000", "barometer": "1015.612", "radiation": "628.00", "wind_samples": "24.0", "inTemp": "27.2", "inHumidity": "49.0", "outHumidity": "62.0", "windSpeed": "2.24", "windGust": "3.58", "windGustDir": "22.50", "windDir": "22.50", "UV": "4.3", "ET": "0.000", "highRadiation": "628.0", "highUV": "4.3", "forecastRule": "45.0", "usUnits": "16.0", "pressure": "988.631", "altimeter": "1017.189", "appTemp": "28.7", "beaufort": "2.0", "cloudbase": "1204.0", "dewpoint": "19.1", "heatindex": "28.2", "humidex": "33.8", "inDewpoint": "15.6", "windchill": "27.0", "windrun": "0.1", "hourRain": "0.000", "rain24": "0.000", "dayRain": "0.000"}

[imbeacon]

There is no parameter temp in the incoming message, so it cannot find the value of this parameter.

If incoming message contains the value from key, the key on the ThingsBoard will be a value. Just use another one value for the key parameter.

[mirtaheri]

There is no parameter temp in the incoming message, so it cannot find the value of this parameter.

If incoming message contains the value from key, the key on the ThingsBoard will be a value. Just use another one value for the key parameter.

yes, it is outTemp. First example I mentioned above, was taken from TB official page's example, just to show the structure of what I was referring to. Please dismiss that and consider the complete one below that (MQTT Connector) which is the real one. Thanks.

[imbeacon]

Please try the following config, I have just add the symbol '_' but it will show it on the UI:

{
  "broker": {
    "name": "WeatherStationBroker",
    "host": "127.0.0.1",
    "port": 1883,
    "security": {
      "type": "anonymous"
    }
  },
  "mapping": [
    {
      "topicFilter": "WeatherSensors/loop",
      "converter": {
        "type": "json",
    "deviceNameJsonExpression": "WeatherStationSensor",
        "deviceTypeJsonExpression": "Sensors",
        "timeout": 60000,
        "timeseries": [
          {
            "type": "double",
            "key": "outTemp_",
            "value": "${outTemp}"
          },
          {
            "type": "double",
            "key": "highOutTemp_",
            "value": "${highOutTemp}"
          },
          {
            "type": "double",
            "key": "lowOutTemp_",
            "value": "${lowOutTemp}"
          },
          {
            "type": "double",
            "key": "rain_",
            "value": "${rain}"
          },
          {
            "type": "double",
            "key": "rainRate_",
            "value": "${rainRate}"
          },
          {
            "type": "double",
            "key": "barometer_",
            "value": "${barometer}"
          },
          {
            "type": "double",
            "key": "radiation_",
            "value": "${radiation}"
          },
          {
            "type": "double",
            "key": "inTemp_",
            "value": "${inTemp}"
          },
          {
            "type": "double",
            "key": "inHumidity_",
            "value": "${inHumidity}"
          }
        ]
      }
    }
  ],
  "connectRequests": [],
  "disconnectRequests": [],
  "attributeUpdates": [],
  "serverSideRpc": [],
  "attributeRequests": []
}

[mirtaheri]

Yes, I tried it and it is working in this way:

I also waited to be sure the keys are updated and it is succeded. We should solve the problem in this way or it was only a test to figuring it out?

[imbeacon]

I will write the fix for this issue and it will parse the value as a key only when it will have ${} in the value of the key. Thank you for your report. Currently, you can fix it by this way.

[mirtaheri]

I will write the fix for this issue and it will parse the value as a key only when it will have ${} in the value of the key. Thank you for your report. Currently, you can fix it by this way.

Thank you very much, so kind of you!

[imbeacon]

Could you check - which version are you use?

[mirtaheri]

Could you check - which version are you use?

It is 2.5.1

[MrPivato]

I'm having the same issue. Any updates on the matter?

[imbeacon]

Hi @mirtaheri @MrPivato

I fixed it today in the master branch, you can try to use the version from it to solve the issue. It was an error in the uplink converter.

[MrPivato]

Thanks! I will try it and let you know.

[naddush]

Hi, I'm new to using thingsboard gateway, I've just installed it but I don't know ito change in the config file of MQTT connector to be able to connect my thingsboard devices to the gateway and send telemetry data, should I change the section "mapping" ?

[imbeacon]

Hi @naddush

Thank you for your interest in the ThingsBoard IoT gateway. You can find the documentation for the MQTT connector here.

[naddush]

hi @zbeacon thank you for your reply, I've actually followed the steps in this doc but still not getting results, even though I changer my mqtt broker port and used the same config file for mapping just for test but I think that it's not connected. here's attached my mqtt.json config file:

[imbeacon]

Hi @naddush ,

I see device with name "SN-001" - It looks like device, created by the gateway, please check the telemetry in this device instead of checking the gateway telemetry.

The gateway creates devices in ThingsBoard instance.

[imbeacon]

Hi @Nadou14 ,

To connect devices, you can follow this guide (For MQTT). Also we have different protocols which are supported by the gateway like Modbus, OPC-UA, etc. Guides for them you also can find on our site.

[naddush]

Hi @naddush ,

I see device with name "SN-001" - It looks like device, created by the gateway, please check the telemetry in this device instead of checking the gateway telemetry.

The gateway creates devices in ThingsBoard instance.

Thank you @zbeacon for your reply, Actually I checked in the device telemetry but any data was received, Although when I checked in the rule engine log files, I see that my data was published from my device, but not received by the gateway

I don't know where the problem is coming from because my gateway is connected to thingsboard and also my broker is configured,

[imbeacon]

Hi @naddush ,

You have wrong configuration to get device name from topic instead of getting from the incoming message, please looking for deviceNameTopicExpression parameter in the article, I sent earlier. You can use it in your case. Or put the serialNumber parameter with device name into the every message from the device and send data to the topic /sensors