thingsboard / thingsboard-gateway

Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols
https://thingsboard.io/docs/iot-gateway/what-is-iot-gateway/
Apache License 2.0
1.74k stars 844 forks source link

[HELP] Thingsboard IoT Gateway , How to override thingsboard timestamp? #1506

Open KershG opened 2 months ago

KershG commented 2 months ago

Describe the issue With regards to Thingsboard IoT Gateway. I have setup an FTP connector to read my excel data. Is it possible to read my timestamp data and overwrite the thingsboard timestamp. I included ts but it doesnt seem to overwrite. Is it possible to do this? My aim to extract a timestamp column from my csv file to assign a ts for each key value pair. Any assistance would be appreciated.

Configuration (ftp.json) { "host": "0.0.0.0", "port": 21, "TLSSupport": false, "security": { "type": "basic", "username": "username", "password": "password" }, "paths": [ { "devicePatternName": "${ReconArea}", "devicePatternType": "Device", "delimiter": ",", "path": "/home/ftpuser/ftp/files/final_output_with_energy_nodes.csv", "readMode": "FULL", "maxFileSize": 15, "pollPeriod": 60, "txtFileDataView": "TABLE", "useHeader": true, "withSortingFiles": true,

"attributes": [ ], "timeseries": [ { "ts": "1724050000", "type": "string", "key": "${MeterNumber}", "value": "${Units}" } ] }

**Connector name FTP Connector

Versions (please complete the following information):

samson0v commented 2 months ago

Hi @KershG, thanks for your interest in ThingsBoard IoT Gateway! Can you provide the connector logs (change level log from INFO to DEBUG)?

KershG commented 2 months ago

Thanks, i changed it to DEBUG but my connector log still seems to be empty. This is the only logs coming from the journalctl. My devices and telemetry still gets uploaded as normal. Did I miss a config step as I do not see anything in connector logs?

Aug 19 12:01:24 tb-gateway2 systemd[1]: Started ThingsBoard Gateway. Aug 19 12:01:24 tb-gateway2 python3[9906]: Cannot load GRPC connector! Aug 19 12:01:24 tb-gateway2 python3[9906]: 2024-08-19 12:01:24 - |INFO| - [tb_gateway_service.py] - tb_gateway_service - init - 225 - Gateway starting... Aug 19 12:01:24 tb-gateway2 python3[9906]: 2024-08-19 12:01:24 - |INFO| - [tb_gateway_service.py] - tb_gateway_service - init - 228 - ThingsBoard IoT gateway version: 3.5.1 Aug 19 12:01:25 tb-gateway2 python3[9906]: 2024-08-19 12:01:25 - |INFO| - [tb_device_mqtt.py] - tb_device_mqtt - _on_connect - 352 - MQTT client <paho.mqtt.client.Client object at 0x7f16f5f510d0> - Connected! Aug 19 12:01:25 tb-gateway2 python3[9906]: 2024-08-19 12:01:25 - |INFO| - [tb_gateway_mqtt.py] - tb_gateway_mqtt - gw_subscribe_to_attribute - 236 - Subscribed to | with id 1 for device * Aug 19 12:01:26 tb-gateway2 python3[9906]: 2024-08-19 12:01:26 - |INFO| - [tb_gateway_service.py] - tb_gateway_service - init - 284 - Gateway started. Aug 19 12:01:26 tb-gateway2 python3[9906]: 2024-08-19 12:01:26 - |INFO| - [tb_loader.py] - tb_loader - import_module - 68 - Import FTPConnector from /usr/lib/python3/dist-packages/thingsboard_gateway/connectors/ftp. Aug 19 12:01:26 tb-gateway2 python3[9906]: 2024-08-19 12:01:26 - |INFO| - [ftp_connector.py] - ftp_connector - init - 80 - FTP Connector started. Aug 19 12:01:26 tb-gateway2 python3[9906]: 2024-08-19 12:01:26 - |INFO| - [ftp_connector.py] - ftp_connector - connect - 128 - Connected to FTP server Aug 19 12:01:26 tb-gateway2 python3[9906]: 2024-08-19 12:01:26 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - init__ - 66 - Remote Configurator started Aug 19 12:01:26 tb-gateway2 python3[9906]: 2024-08-19 12:01:26 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 204 - Configuration update request received. Aug 19 12:01:26 tb-gateway2 python3[9906]: 2024-08-19 12:01:26 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 204 - Configuration update request received.