[HELP]

[thaxir]

Describe the issue Create description about your issue, and your actions to solve it.

Configuration (Attach your configuration file) Notate: Remove Access token from file if you wanna attach tb_gateway.yaml

Component Thingsboard-gateway , TB-PE

Generic

Description Hello!, I have a TB-PE connected with a TB-Gw and through mqtt i receive messages like the following: mosquitto_pub -d -q 1 -h localhost -p "1883" -t "/sensor/data" -u "*****" -m '{"unit": "%", "description": "state of charge", "status": 0, "value": null, "device_name": "victron", "timestamp": "2021-09-14T06:32:07.507569+00:00", "sensor_type": "state_of_charge", "scalefactor": 1.0, "smart_device": "RPi001"}'

When i send that timestamp (e.g "timestamp": "2021-09-14T06:32:07.507569+00:00" ) TB shows nothing.

1. My first question is how can i get this timestamp field correctly in order for TB to be able to understand it and sort it (if i use ts field in order to override server's timestamp i again cannot see it on TB) ?
2. Is the integration option of TB a better choice to have here? If not what is the difference between IOT gw and integration feature

Just to say i use the default converter of TB-GW (see mqtt.json below)

Mqtt Json { "broker": { "name":"Default Local Broker", "host":"127.0.0.1", "port":1883, "clientId": "ThingsBoard_gateway", "security": { "type": "basic", "username": "username", "password": "password" } }, "mapping": [ { "topicFilter": "/sensor/data", "converter": { "type": "json", "deviceNameJsonExpression": "${serialNumber}", "deviceTypeJsonExpression": "${sensorType}", "timeout": 60000, "attributes": [ { "type": "string", "key": "model", "value": "${sensorModel}" }, { "type": "string", "key": "${sensorModel}", "value": "on" } ], "timeseries": [ { "type": "double", "key": "temperature", "value": "${temperature}" }, { "type": "double", "key": "humidity", "value": "${hum}" } ] } }, { "topicFilter": "/sensor/+/data", "converter": { "type": "json", "deviceNameTopicExpression": "(?<=sensor/)(.?)(?=/data)", "deviceTypeTopicExpression": "Thermometer", "timeout": 60000, "attributes": [ { "type": "string", "key": "model", "value": "${sensorModel}" } ], "timeseries": [ { "type": "double", "key": "temperature", "value": "${temp}" }, { "type": "double", "key": "humidity", "value": "${hum}" } ] } }, { "topicFilter": "/custom/sensors/+", "converter": { "type": "custom", "extension": "CustomMqttUplinkConverter", "extension-config": { "temperatureBytes" : 2, "humidityBytes" : 2, "batteryLevelBytes" : 1 } } } ], "connectRequests": [ { "topicFilter": "sensor/connect", "deviceNameJsonExpression": "${SerialNumber}" }, { "topicFilter": "sensor/+/connect", "deviceNameTopicExpression": "(?<=sensor/)(.?)(?=/connect)" } ], "disconnectRequests": [ { "topicFilter": "sensor/disconnect", "deviceNameJsonExpression": "${SerialNumber}" }, { "topicFilter": "sensor/+/disconnect", "deviceNameTopicExpression": "(?<=sensor/)(.?)(?=/disconnect)" } ], "attributeUpdates": [ { "deviceNameFilter": "SmartMeter.", "attributeFilter": "uploadFrequency", "topicExpression": "sensor/${deviceName}/${attributeKey}", "valueExpression": "{"${attributeKey}":"${attributeValue}"}" } ], "serverSideRpc": [ { "deviceNameFilter": ".", "methodFilter": "echo", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}", "responseTimeout": 10000, "valueExpression": "${params}" }, { "deviceNameFilter": ".", "methodFilter": "no-reply", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "valueExpression": "${params}" } ] } Environment

OS: Ubuntu 18.04
ThingsBoard: latest TB-PE
ThingsBoard Gateway : 2.9
Browser: firefox

[imbeacon]

Hi @Kernelalive ,

Thank you for your interest in ThingsBoard IoT gateway.

My first question is how can i get this timestamp field correctly in order for TB to be able to understand it and sort it (if i use ts field in order to override server's timestamp i again cannot see it on TB) ?

"ts" or "timestamp" parameter should contain long value with milliseconds since 01.01.1970 e.g. 1636640012783 for Thursday, November 11, 2021 2:13:32.783 PM

Is the integration option of TB a better choice to have here? If not what is the difference between IOT gw and integration feature

The gateway and integrations contains almost the same functionality, but integrations can be used only by users of PE version and they are able to process more datapoints per second then the gateway. Because at the moment, the gateway can work only with one core. But at the same time, the gateway has different storage types and they may be preferable in some cases.

If you are using PE version, you can test the both variants and decide what is the better for your case.

[thaxir]

Hi @zbeacon , and thank you for your answer. So in continuation to this task and in order to get the timestamp of the client I have tried to override the timestamp by sending this type of message : mosquitto_pub -d -q 1 -h localhost -p "1883" -t "/sensor/data" -u "username" -P "password" -m '{"serialNumber":"Device A", "sensorType":"thermostat", "sensorModel":"Model A","temperature":4222,"hum":12222323232323,"timestamp":"2021-11-12T11:47:07.507569+00:00"}'

with the default mqtt.json . I get no messages on the TB. See the logs below. This happens when i add an extra field in the mqtt.json in the telemetry section like and instead of ts i put timestamp key so to match the name with the config below. { "type": "long", "key": "time", "value": "${timestamp}" }

service.log connector.log

tb_connection.log

[imbeacon]

It is a wrong message, the gateway cannot parse it. expected value is integer instead of string. The correct command with message is: mosquitto_pub -d -q 1 -h localhost -p "1883" -t "/sensor/data" -u "username" -P "password" -m '{"serialNumber":"Device A", "sensorType":"thermostat", "sensorModel":"Model A","temperature":4222,"hum":12222323232323,"timestamp":1636717627507}'

[imbeacon]

Also String "2021-11-12T11:47:07.507569+00:00" cannot be converted to long.

[thaxir]

@zbeacon ok and how can modify the timestamp to the format you propose(unix time)? Can this be done on the fly i.e on the mqtt.json of iot Gw or i need different approach?

[rduivenvoorde]

@Kernelalive we also use the gateway, but sent both 'ts' AND some stringbased values in the messages, ready for the gateway to push into some attribute or variable. EG:

{...., "ts": 1637577481113, "time_utc": "2021-11-22 10:38:01Z", "time_local": "2021-11-22 11:38:01"}

'ts' then is used by Thingsboard as part of telemetry value (you can even leave it off, then telemetry will be based on time of message), and the other string timestamps I use for showing in other parts of the dashboards.

[thaxir]

Hi @rduivenvoorde , thank you for the reply I want to override the time of the server with the time i get from clients .This is because i need to do the sorting in terms of time based on what time the client sent .If is it possible to do such a thing without overriding the original ts that the server assigns this is also something i could do. The problem on my side is that i cannot change this type 2021-11-12T11:47:07.507569+00:00 of format to a unix time . I actually face problem parsing the whole value 2021-11-12T11:47:07.507569+00:00 which returns me null while parsing this 2021-11-12T11:47:07 is working. So either if i can sort a different timestamp field or fix this by transforming the value i wrote before to unix time solves my issue.

[samson0v]

Hi @Kernelalive, can we close the issue?