search

thingsboard / thingsboard-gateway

Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols
https://thingsboard.io/docs/iot-gateway/what-is-iot-gateway/
Apache License 2.0
1.76k stars 845 forks source link

[BUG] Sliced ​​and Table FTP Connector broken with ts #1581

Closed jean-louis closed 2 weeks ago

jean-louis commented 2 weeks ago

Describe the bug Using the ftp connector in sliced ​​or table mode, you get an error if the ts attribute is present. The device is not created and the data is not sent to the ThingsBoard (cloud) platform.

Attached are the configuration files and related logs.

I will open a PR with the fixes I am using locally.

Connector name: FTP Connector

Error traceback:

tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()
tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()
tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()
tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()
tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()

Versions (please complete the following information):

sliced_connector.json sliced_connector.log table_connector.json table_connector.log

jean-louis commented 2 weeks ago

test_ftp_bug_sliced.txt test_ftp_bug_table.csv

jean-louis commented 2 weeks ago

With the code in master branch, the error is no longer there, but the custom ts is not taken into account.

PR https://github.com/thingsboard/thingsboard-gateway/pull/1582 fixes this problem

imbeacon commented 2 weeks ago

Hi @jean-louis,

Please try the version from the master branch, I also have checked your PR and should notice that it will properly work only if ts key configuration will be the very first in timeseries configuration array. Please try our variant of implementation and let us know about the results or any issues.

jean-louis commented 2 weeks ago

Hi @imbeacon ,

with te new commit on master branch I get this error:

For sliced

tb-gateway  | 2024-11-06 10:28:12 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 227 - Configuration update request received.
tb-gateway  | 2024-11-06 10:28:12 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 227 - Configuration update request received.
tb-gateway  | 2024-11-06 10:28:12 - |INFO| - [tb_loader.py] - tb_loader - import_module - 69 - Import FTPConnector from /thingsboard_gateway/connectors/ftp.
tb-gateway  | 2024-11-06 10:28:12 - |INFO| - [ftp_connector.py] - ftp_connector - __init__ - 83 - FTP Connector started.
tb-gateway  | 2024-11-06 10:28:12 - |INFO| - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 1001 - Cannot find separated report strategy for connector 'Test ftp bug - sliced - Connector'. The main report strategy will be used as a connector report strategy.
tb-gateway  | 2024-11-06 10:28:12 - |INFO| - [ftp_connector.py] - ftp_connector - __connect - 131 - Connected to FTP server
tb-gateway  | 2024-11-06 10:28:12 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 227 - Configuration update request received.
tb-gateway  | 2024-11-06 10:28:12 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_slices_view_data - 164 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - sliced new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "[0:1]"}, {"type": "int", "key": "x1", "value": "[1:2]"}, {"type": "int", "key": "x2", "value": "[2:3]"}, {"type": "int", "key": "x3", "value": "[3:4]"}, {"type": "int", "key": "x4", "value": "[4:5]"}, {"type": "int", "key": "x5", "value": "[5:6]"}], "attributes": [], "txt_file_data_view": "SLICED", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069221,11,12,13,14,15
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:28:12 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - sliced new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:28:12 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_slices_view_data - 164 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - sliced new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "[0:1]"}, {"type": "int", "key": "x1", "value": "[1:2]"}, {"type": "int", "key": "x2", "value": "[2:3]"}, {"type": "int", "key": "x3", "value": "[3:4]"}, {"type": "int", "key": "x4", "value": "[4:5]"}, {"type": "int", "key": "x5", "value": "[5:6]"}], "attributes": [], "txt_file_data_view": "SLICED", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069267,21,22,23,24,25
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:28:12 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - sliced new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:28:12 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_slices_view_data - 164 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - sliced new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "[0:1]"}, {"type": "int", "key": "x1", "value": "[1:2]"}, {"type": "int", "key": "x2", "value": "[2:3]"}, {"type": "int", "key": "x3", "value": "[3:4]"}, {"type": "int", "key": "x4", "value": "[4:5]"}, {"type": "int", "key": "x5", "value": "[5:6]"}], "attributes": [], "txt_file_data_view": "SLICED", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069313,31,32,33,34,35
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:28:12 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - sliced new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:28:12 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_slices_view_data - 164 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - sliced new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "[0:1]"}, {"type": "int", "key": "x1", "value": "[1:2]"}, {"type": "int", "key": "x2", "value": "[2:3]"}, {"type": "int", "key": "x3", "value": "[3:4]"}, {"type": "int", "key": "x4", "value": "[4:5]"}, {"type": "int", "key": "x5", "value": "[5:6]"}], "attributes": [], "txt_file_data_view": "SLICED", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069359,41,42,43,44,45
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:28:12 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - sliced new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:28:12 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_slices_view_data - 164 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - sliced new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "[0:1]"}, {"type": "int", "key": "x1", "value": "[1:2]"}, {"type": "int", "key": "x2", "value": "[2:3]"}, {"type": "int", "key": "x3", "value": "[3:4]"}, {"type": "int", "key": "x4", "value": "[4:5]"}, {"type": "int", "key": "x5", "value": "[5:6]"}], "attributes": [], "txt_file_data_view": "SLICED", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069405,51,52,53,54,55
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:28:12 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - sliced new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:28:12 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: None
tb-gateway  | 2024-11-06 10:37:36 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 227 - Configuration update request received.
tb-gateway  | 2024-11-06 10:37:36 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 227 - Configuration update request received.
tb-gateway  | 2024-11-06 10:37:36 - |INFO| - [ftp_connector.py] - ftp_connector - close - 202 - FTP Connector stopped.

For table:

tb-gateway  | 2024-11-06 10:39:16 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 227 - Configuration update request received.
tb-gateway  | 2024-11-06 10:39:16 - |INFO| - [ftp_connector.py] - ftp_connector - __init__ - 83 - FTP Connector started.
tb-gateway  | 2024-11-06 10:39:16 - |INFO| - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 1001 - Cannot find separated report strategy for connector 'Test ftp bug - table - Connector'. The main report strategy will be used as a connector report strategy.
tb-gateway  | 2024-11-06 10:39:16 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 227 - Configuration update request received.
tb-gateway  | 2024-11-06 10:39:16 - |INFO| - [ftp_connector.py] - ftp_connector - __connect - 131 - Connected to FTP server
tb-gateway  | 2024-11-06 10:39:17 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_table_view_data - 108 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - table new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "${ts}"}, {"type": "int", "key": "x1", "value": "${x1}"}, {"type": "int", "key": "x2", "value": "${x2}"}, {"type": "int", "key": "x3", "value": "${x3}"}, {"type": "int", "key": "x4", "value": "${x4}"}, {"type": "int", "key": "x5", "value": "${x5}"}], "attributes": [], "txt_file_data_view": "TABLE", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069221,11,12,13,14,15
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:39:17 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:39:17 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_table_view_data - 108 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - table new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "${ts}"}, {"type": "int", "key": "x1", "value": "${x1}"}, {"type": "int", "key": "x2", "value": "${x2}"}, {"type": "int", "key": "x3", "value": "${x3}"}, {"type": "int", "key": "x4", "value": "${x4}"}, {"type": "int", "key": "x5", "value": "${x5}"}], "attributes": [], "txt_file_data_view": "TABLE", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069267,21,22,23,24,25
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:39:17 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:39:17 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_table_view_data - 108 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - table new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "${ts}"}, {"type": "int", "key": "x1", "value": "${x1}"}, {"type": "int", "key": "x2", "value": "${x2}"}, {"type": "int", "key": "x3", "value": "${x3}"}, {"type": "int", "key": "x4", "value": "${x4}"}, {"type": "int", "key": "x5", "value": "${x5}"}], "attributes": [], "txt_file_data_view": "TABLE", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069313,31,32,33,34,35
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:39:17 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:39:17 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_table_view_data - 108 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - table new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "${ts}"}, {"type": "int", "key": "x1", "value": "${x1}"}, {"type": "int", "key": "x2", "value": "${x2}"}, {"type": "int", "key": "x3", "value": "${x3}"}, {"type": "int", "key": "x4", "value": "${x4}"}, {"type": "int", "key": "x5", "value": "${x5}"}], "attributes": [], "txt_file_data_view": "TABLE", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069359,41,42,43,44,45
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:39:17 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:39:17 - |ERROR| - [ftp_uplink_converter.py] - ftp_uplink_converter - _convert_table_view_data - 108 - Error in converter, for config: 
tb-gateway  | {"delimiter": ",", "devicePatternName": "Test ftp bug - table new master", "devicePatternType": "Device", "timeseries": [{"type": "int", "key": "ts", "value": "${ts}"}, {"type": "int", "key": "x1", "value": "${x1}"}, {"type": "int", "key": "x2", "value": "${x2}"}, {"type": "int", "key": "x3", "value": "${x3}"}, {"type": "int", "key": "x4", "value": "${x4}"}, {"type": "int", "key": "x5", "value": "${x5}"}], "attributes": [], "txt_file_data_view": "TABLE", "reportStrategy": null}
tb-gateway  |  and message: 
tb-gateway  | 1726830069405,51,52,53,54,55
tb-gateway  |  list indices must be integers or slices, not str
tb-gateway  | 2024-11-06 10:39:17 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master, deviceType=Device, telemetry=[], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 10:39:17 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 187 - Converted data: None
tb-gateway  | 2024-11-06 10:39:17 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 227 - Configuration update request received.
imbeacon commented 2 weeks ago

Hi @jean-louis, Yes, my bad, passed wrong configuration object to parsing, please try now.

jean-louis commented 2 weeks ago

Hi @imbeacon ,

now sliced works fine, but table does not correctly take the value of ts:

tb-gateway  | 2024-11-06 11:46:50 - |INFO| - [ftp_connector.py] - ftp_connector - __connect - 131 - Connected to FTP server
tb-gateway  | 2024-11-06 11:46:50 - |INFO| - [ftp_connector.py] - ftp_connector - __process_paths - 202 - Converted data for device Test ftp bug - table new master 2 with type Device, attributes: 0, telemetry: 5
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 206 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '11', DatapointKey(key=x2, report_strategy=None): '12', DatapointKey(key=x3, report_strategy=None): '13', DatapointKey(key=x4, report_strategy=None): '14', DatapointKey(key=x5, report_strategy=None): '15'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __send_data - 217 - Data to ThingsBoard: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '11', DatapointKey(key=x2, report_strategy=None): '12', DatapointKey(key=x3, report_strategy=None): '13', DatapointKey(key=x4, report_strategy=None): '14', DatapointKey(key=x5, report_strategy=None): '15'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:50 - |INFO| - [ftp_connector.py] - ftp_connector - __process_paths - 202 - Converted data for device Test ftp bug - table new master 2 with type Device, attributes: 0, telemetry: 5
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 206 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '21', DatapointKey(key=x2, report_strategy=None): '22', DatapointKey(key=x3, report_strategy=None): '23', DatapointKey(key=x4, report_strategy=None): '24', DatapointKey(key=x5, report_strategy=None): '25'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __send_data - 217 - Data to ThingsBoard: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '21', DatapointKey(key=x2, report_strategy=None): '22', DatapointKey(key=x3, report_strategy=None): '23', DatapointKey(key=x4, report_strategy=None): '24', DatapointKey(key=x5, report_strategy=None): '25'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:50 - |INFO| - [ftp_connector.py] - ftp_connector - __process_paths - 202 - Converted data for device Test ftp bug - table new master 2 with type Device, attributes: 0, telemetry: 5
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 206 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '31', DatapointKey(key=x2, report_strategy=None): '32', DatapointKey(key=x3, report_strategy=None): '33', DatapointKey(key=x4, report_strategy=None): '34', DatapointKey(key=x5, report_strategy=None): '35'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __send_data - 217 - Data to ThingsBoard: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '31', DatapointKey(key=x2, report_strategy=None): '32', DatapointKey(key=x3, report_strategy=None): '33', DatapointKey(key=x4, report_strategy=None): '34', DatapointKey(key=x5, report_strategy=None): '35'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:50 - |INFO| - [ftp_connector.py] - ftp_connector - __process_paths - 202 - Converted data for device Test ftp bug - table new master 2 with type Device, attributes: 0, telemetry: 5
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 206 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '41', DatapointKey(key=x2, report_strategy=None): '42', DatapointKey(key=x3, report_strategy=None): '43', DatapointKey(key=x4, report_strategy=None): '44', DatapointKey(key=x5, report_strategy=None): '45'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __send_data - 217 - Data to ThingsBoard: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '41', DatapointKey(key=x2, report_strategy=None): '42', DatapointKey(key=x3, report_strategy=None): '43', DatapointKey(key=x4, report_strategy=None): '44', DatapointKey(key=x5, report_strategy=None): '45'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:50 - |INFO| - [ftp_connector.py] - ftp_connector - __process_paths - 202 - Converted data for device Test ftp bug - table new master 2 with type Device, attributes: 0, telemetry: 5
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __process_paths - 206 - Converted data: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '51', DatapointKey(key=x2, report_strategy=None): '52', DatapointKey(key=x3, report_strategy=None): '53', DatapointKey(key=x4, report_strategy=None): '54', DatapointKey(key=x5, report_strategy=None): '55'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:50 - |DEBUG| - [ftp_connector.py] - ftp_connector - __send_data - 217 - Data to ThingsBoard: ConvertedData(deviceName=Test ftp bug - table new master 2, deviceType=Device, telemetry=[TelemetryEntry(ts=${ts}, metadata={}, values={DatapointKey(key=x1, report_strategy=None): '51', DatapointKey(key=x2, report_strategy=None): '52', DatapointKey(key=x3, report_strategy=None): '53', DatapointKey(key=x4, report_strategy=None): '54', DatapointKey(key=x5, report_strategy=None): '55'})], attributes=Attributes(values={}), metadata={})
tb-gateway  | 2024-11-06 11:46:51 - |INFO| - [tb_gateway_remote_configurator.py] - tb_gateway_remote_configurator - _process_config_request - 227 - Configuration update request received.

you can see:

telemetry=[TelemetryEntry(ts=${ts}
imbeacon commented 2 weeks ago

Yes, please find fix in https://github.com/thingsboard/thingsboard-gateway/commit/b7f59b896f92b5ed15026b870a17effbe66ffcb4

jean-louis commented 2 weeks ago

ok, now both work fine

thanks for your work @imbeacon

I will close the PR because it is no longer necessary