thingsboard / thingsboard-gateway

Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols
https://thingsboard.io/docs/iot-gateway/what-is-iot-gateway/
Apache License 2.0
1.75k stars 844 forks source link

[BUG] Service Cannot Start When Using Modbus Connector from Latest Release v3.2 Debian Installer #977

Closed helsmore closed 2 years ago

helsmore commented 2 years ago

Describe the bug The systemd service installed by the v3.2 Debian Installer fails at startup once the installation is completed and again when the gateway configuration has been setup.

I suspect that the cause of the failure is that the extension folders are getting placed (by the installer) directly into /var/lib/thingsboard_gateway which is the working directory used when starting the service (rather than placing them in say an extensions sub-directory), this means that import statements for (e.g.) socket and serial are resolving to the gateway extensions with these names thus shadowing the actual system packages with the same names and causing attribute errors and service restarts/failures.

This seems very similar to a problem reported in a comment on a previous issue originally posted in https://github.com/thingsboard/thingsboard-gateway/issues/839#issuecomment-1163379504 by @ilseva and @wilterdinkrobert and indeed that's where I got the workaround listed for the first error from.

Connector name (If bug in the some connector): Modbus, but entire service cannot start even without the Modbus connector active.

Error traceback (If available): For error relating to socket, occurred immediately after installation completed:

user@host:~ $  journalctl --follow -u thingsboard-gateway.service
-- Journal begins at Tue 2022-09-06 20:06:18 BST. --
Nov 01 12:57:56 raspberrypi-iot-gw python3[5583]:   File "/usr/lib/python3.9/socketserver.py", line 437, in TCPServer
Nov 01 12:57:56 raspberrypi-iot-gw python3[5583]:     address_family = socket.AF_INET
Nov 01 12:57:56 raspberrypi-iot-gw python3[5583]: AttributeError: module 'socket' has no attribute 'AF_INET'
Nov 01 12:57:56 raspberrypi-iot-gw systemd[1]: thingsboard-gateway.service: Main process exited, code=exited, status=1/FAILURE
Nov 01 12:57:56 raspberrypi-iot-gw systemd[1]: thingsboard-gateway.service: Failed with result 'exit-code'.
Nov 01 12:57:56 raspberrypi-iot-gw systemd[1]: thingsboard-gateway.service: Scheduled restart job, restart counter is at 7.
Nov 01 12:57:56 raspberrypi-iot-gw systemd[1]: Stopped ThingsBoard Gateway.
Nov 01 12:57:56 raspberrypi-iot-gw systemd[1]: thingsboard-gateway.service: Start request repeated too quickly.
Nov 01 12:57:56 raspberrypi-iot-gw systemd[1]: thingsboard-gateway.service: Failed with result 'exit-code'.
Nov 01 12:57:56 raspberrypi-iot-gw systemd[1]: Failed to start ThingsBoard Gateway.
^C
user@host:~ $

The workaround for this error to allow me to get the service to start at all was as follows:

user@host:~ $ sudo mv /var/lib/thingsboard_gateway/socket /var/lib/thingsboard_gateway/mysocket
user@host:~ $ sudo systemctl restart thingsboard-gateway.service

After the above workaround I modified the configuration to point to my TB platform instance and activated the Modbus connector, I then got the following error relating to PyModbus importing the system serial module:

user@host:~ $ journalctl --follow -u thingsboard-gateway.service
-- Journal begins at Tue 2022-09-06 20:06:18 BST. --
Nov 01 13:07:19 raspberrypi-iot-gw python3[5677]: ""2022-11-01 13:07:19" - |ERROR| - [mqtt_connector.py] - mqtt_connector - __connect - 285 - [Errno 111] Connection refused"
Nov 01 13:07:23 raspberrypi-iot-gw systemd[1]: Stopping ThingsBoard Gateway...
Nov 01 13:07:23 raspberrypi-iot-gw python3[5677]: ""2022-11-01 13:07:23" - |INFO| - [tb_gateway_service.py] - tb_gateway_service - __stop_gateway - 355 - Stopping..."
Nov 01 13:07:23 raspberrypi-iot-gw python3[5677]: ""2022-11-01 13:07:23" - |INFO| - [mqtt_connector.py] - mqtt_connector - close - 295 - MQTT Broker Connector has been stopped."
Nov 01 13:07:23 raspberrypi-iot-gw python3[5677]: ""2022-11-01 13:07:23" - |INFO| - [tb_gateway_service.py] - tb_gateway_service - __stop_gateway - 364 - The gateway has been stopped."
Nov 01 13:07:23 raspberrypi-iot-gw python3[5677]: ""2022-11-01 13:07:23" - |INFO| - [tb_gateway_mqtt.py] - tb_gateway_mqtt - gw_unsubscribe - 193 - Unsubscribed from *|*, subscription id '*'"
Nov 01 13:07:23 raspberrypi-iot-gw systemd[1]: thingsboard-gateway.service: Succeeded.
Nov 01 13:07:23 raspberrypi-iot-gw systemd[1]: Stopped ThingsBoard Gateway.
Nov 01 13:07:23 raspberrypi-iot-gw systemd[1]: thingsboard-gateway.service: Consumed 3.822s CPU time.
Nov 01 13:07:23 raspberrypi-iot-gw systemd[1]: Started ThingsBoard Gateway.
Nov 01 13:07:25 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:07:25" - |INFO| - [tb_gateway_service.py] - tb_gateway_service - __init__ - 148 - Gateway starting..."
Nov 01 13:07:25 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:07:25" - |INFO| - [tb_gateway_service.py] - tb_gateway_service - __init__ - 153 - ThingsBoard IoT gateway version: 3.2"
Nov 01 13:07:25 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:07:25" - |INFO| - [tb_gateway_mqtt.py] - tb_gateway_mqtt - gw_subscribe_to_attribute - 185 - Subscribed to *|* with id 1 for device *"
Nov 01 13:07:33 raspberrypi-iot-gw python3[5830]: Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Nov 01 13:07:35 raspberrypi-iot-gw python3[5830]: Collecting pymodbus>=3.0.0
Nov 01 13:07:35 raspberrypi-iot-gw python3[5830]:   Downloading https://www.piwheels.org/simple/pymodbus/pymodbus-3.0.0-py2.py3-none-any.whl (137 kB)
Nov 01 13:07:38 raspberrypi-iot-gw python3[5830]: Installing collected packages: pymodbus
Nov 01 13:07:38 raspberrypi-iot-gw python3[5830]:   WARNING: The scripts pymodbus.console and pymodbus.server are installed in '/var/lib/thingsboard_gateway/.local/bin' which is not on PATH.
Nov 01 13:07:38 raspberrypi-iot-gw python3[5830]:   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Nov 01 13:07:38 raspberrypi-iot-gw python3[5830]: Successfully installed pymodbus-3.0.0
Nov 01 13:07:48 raspberrypi-iot-gw python3[5834]: Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Nov 01 13:07:56 raspberrypi-iot-gw python3[5834]: Collecting pyserial
Nov 01 13:07:57 raspberrypi-iot-gw python3[5834]:   Downloading https://www.piwheels.org/simple/pyserial/pyserial-3.5-py2.py3-none-any.whl (90 kB)
Nov 01 13:07:59 raspberrypi-iot-gw python3[5834]: Installing collected packages: pyserial
Nov 01 13:07:59 raspberrypi-iot-gw python3[5834]:   WARNING: The scripts pyserial-miniterm and pyserial-ports are installed in '/var/lib/thingsboard_gateway/.local/bin' which is not on PATH.
Nov 01 13:07:59 raspberrypi-iot-gw python3[5834]:   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Nov 01 13:07:59 raspberrypi-iot-gw python3[5834]: Successfully installed pyserial-3.5
Nov 01 13:08:07 raspberrypi-iot-gw python3[5840]: Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Nov 01 13:08:11 raspberrypi-iot-gw python3[5840]: Collecting pyserial-asyncio
Nov 01 13:08:11 raspberrypi-iot-gw python3[5840]:   Downloading https://www.piwheels.org/simple/pyserial-asyncio/pyserial_asyncio-0.6-py3-none-any.whl (7.5 kB)
Nov 01 13:08:11 raspberrypi-iot-gw python3[5840]: Requirement already satisfied: pyserial in ./.local/lib/python3.9/site-packages (from pyserial-asyncio) (3.5)
Nov 01 13:08:12 raspberrypi-iot-gw python3[5840]: Installing collected packages: pyserial-asyncio
Nov 01 13:08:19 raspberrypi-iot-gw python3[5840]: Successfully installed pyserial-asyncio-0.6
Nov 01 13:08:28 raspberrypi-iot-gw python3[5843]: Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Nov 01 13:08:30 raspberrypi-iot-gw python3[5843]: Collecting twisted
Nov 01 13:08:31 raspberrypi-iot-gw python3[5843]:   Downloading https://www.piwheels.org/simple/twisted/Twisted-22.10.0-py3-none-any.whl (3.1 MB)
Nov 01 13:08:36 raspberrypi-iot-gw python3[5843]: Collecting constantly>=15.1
Nov 01 13:08:36 raspberrypi-iot-gw python3[5843]:   Downloading https://www.piwheels.org/simple/constantly/constantly-15.1.0-py2.py3-none-any.whl (7.9 kB)
Nov 01 13:08:37 raspberrypi-iot-gw python3[5843]: Collecting incremental>=21.3.0
Nov 01 13:08:37 raspberrypi-iot-gw python3[5843]:   Downloading https://www.piwheels.org/simple/incremental/incremental-22.10.0-py2.py3-none-any.whl (16 kB)
Nov 01 13:08:37 raspberrypi-iot-gw python3[5843]: Collecting Automat>=0.8.0
Nov 01 13:08:37 raspberrypi-iot-gw python3[5843]:   Downloading https://www.piwheels.org/simple/automat/Automat-22.10.0-py2.py3-none-any.whl (26 kB)
Nov 01 13:08:38 raspberrypi-iot-gw python3[5843]: Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.9/dist-packages (from twisted) (22.1.0)
Nov 01 13:08:39 raspberrypi-iot-gw python3[5843]: Collecting hyperlink>=17.1.1
Nov 01 13:08:39 raspberrypi-iot-gw python3[5843]:   Downloading https://www.piwheels.org/simple/hyperlink/hyperlink-21.0.0-py2.py3-none-any.whl (74 kB)
Nov 01 13:08:42 raspberrypi-iot-gw python3[5843]: Collecting zope.interface>=4.4.2
Nov 01 13:08:42 raspberrypi-iot-gw python3[5843]:   Downloading https://www.piwheels.org/simple/zope-interface/zope.interface-5.5.0-cp39-cp39-linux_armv7l.whl (243 kB)
Nov 01 13:08:49 raspberrypi-iot-gw python3[5843]: Collecting typing-extensions>=3.6.5
Nov 01 13:08:52 raspberrypi-iot-gw python3[5843]:   Downloading https://www.piwheels.org/simple/typing-extensions/typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Nov 01 13:08:55 raspberrypi-iot-gw python3[5843]: Requirement already satisfied: six in /usr/lib/python3/dist-packages (from Automat>=0.8.0->twisted) (1.16.0)
Nov 01 13:08:55 raspberrypi-iot-gw python3[5843]: Requirement already satisfied: idna>=2.5 in /usr/lib/python3/dist-packages (from hyperlink>=17.1.1->twisted) (2.10)
Nov 01 13:08:55 raspberrypi-iot-gw python3[5843]: Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from zope.interface>=4.4.2->twisted) (52.0.0)
Nov 01 13:08:57 raspberrypi-iot-gw python3[5843]: Installing collected packages: zope.interface, typing-extensions, incremental, hyperlink, constantly, Automat, twisted
Nov 01 13:09:00 raspberrypi-iot-gw python3[5843]:   WARNING: The script automat-visualize is installed in '/var/lib/thingsboard_gateway/.local/bin' which is not on PATH.
Nov 01 13:09:00 raspberrypi-iot-gw python3[5843]:   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Nov 01 13:09:15 raspberrypi-iot-gw python3[5843]:   WARNING: The scripts cftp, ckeygen, conch, mailmail, pyhtmlizer, tkconch, trial, twist and twistd are installed in '/var/lib/thingsboard_gateway/.local/bin' which is not on PATH.
Nov 01 13:09:15 raspberrypi-iot-gw python3[5843]:   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Nov 01 13:09:17 raspberrypi-iot-gw python3[5843]: Successfully installed Automat-22.10.0 constantly-15.1.0 hyperlink-21.0.0 incremental-22.10.0 twisted-22.10.0 typing-extensions-4.4.0 zope.interface-5.5.0
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]: Modbus library not found - installing...
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:09:18" - |INFO| - [tb_loader.py] - tb_loader - import_module - 68 - Import ModbusConnector from /usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus."
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:09:18" - |ERROR| - [modbus_connector.py] - modbus_connector - __process_slaves - 342 - module 'serial' has no attribute 'SerialException'"
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 208, in connect
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]:     self.socket = serial.serial_for_url(
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'serial_for_url'
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]: During handling of the above exception, another exception occurred:
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 311, in __process_slaves
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]:     self.__connect_to_current_master(device)
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 387, in __connect_to_current_master
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]:     device.config['master'].connect()
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 220, in connect
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]:     except serial.SerialException as msg:
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'SerialException'
Nov 01 13:09:18 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:09:18" - |INFO| - [tb_gateway_service.py] - tb_gateway_service - __init__ - 252 - Gateway started."
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:09:28" - |ERROR| - [modbus_connector.py] - modbus_connector - __process_slaves - 342 - module 'serial' has no attribute 'SerialException'"
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 208, in connect
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]:     self.socket = serial.serial_for_url(
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'serial_for_url'
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]: During handling of the above exception, another exception occurred:
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 311, in __process_slaves
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]:     self.__connect_to_current_master(device)
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 387, in __connect_to_current_master
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]:     device.config['master'].connect()
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 220, in connect
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]:     except serial.SerialException as msg:
Nov 01 13:09:28 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'SerialException'
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:09:38" - |ERROR| - [modbus_connector.py] - modbus_connector - __process_slaves - 342 - module 'serial' has no attribute 'SerialException'"
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 208, in connect
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]:     self.socket = serial.serial_for_url(
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'serial_for_url'
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]: During handling of the above exception, another exception occurred:
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 311, in __process_slaves
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]:     self.__connect_to_current_master(device)
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 387, in __connect_to_current_master
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]:     device.config['master'].connect()
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 220, in connect
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]:     except serial.SerialException as msg:
Nov 01 13:09:38 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'SerialException'
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:09:43" - |ERROR| - [modbus_connector.py] - modbus_connector - __process_slaves - 342 - module 'serial' has no attribute 'SerialException'"
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 208, in connect
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]:     self.socket = serial.serial_for_url(
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'serial_for_url'
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]: During handling of the above exception, another exception occurred:
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 311, in __process_slaves
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]:     self.__connect_to_current_master(device)
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 387, in __connect_to_current_master
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]:     device.config['master'].connect()
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 220, in connect
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]:     except serial.SerialException as msg:
Nov 01 13:09:43 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'SerialException'
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]: ""2022-11-01 13:09:48" - |ERROR| - [modbus_connector.py] - modbus_connector - __process_slaves - 342 - module 'serial' has no attribute 'SerialException'"
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 208, in connect
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]:     self.socket = serial.serial_for_url(
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'serial_for_url'
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]: During handling of the above exception, another exception occurred:
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]: Traceback (most recent call last):
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 311, in __process_slaves
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]:     self.__connect_to_current_master(device)
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]:   File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 387, in __connect_to_current_master
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]:     device.config['master'].connect()
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]:   File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 220, in connect
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]:     except serial.SerialException as msg:
Nov 01 13:09:48 raspberrypi-iot-gw python3[5821]: AttributeError: module 'serial' has no attribute 'SerialException'
^C
user@host:~ $

The workaround to allow me to get the Modbus connector working was as follows:

user@host:~ $ sudo mv /var/lib/thingsboard_gateway/serial /var/lib/thingsboard_gateway/myserial
user@host:~ $ sudo systemctl restart thingsboard-gateway.service

After this the service worked correctly and data from my Modbus RTU device appeared in my TB platform instance.

Versions (please complete the following information):

Note that to get the service installed on Raspbian at all I had to follow the workaround I've listed in https://github.com/thingsboard/thingsboard-gateway/issues/975#issuecomment-1299919769 as the installer for the Thingsboard IoT Gateway has broken on Raspbian and Debian at the v3.2 release.

samson0v commented 2 years ago

Hi @helsmore, this is an expected situation because you haven't installed pyserial, so Gateway dynamically installs it but Python can't resolve it so you have to restart your Gateway.

samson0v commented 2 years ago

See Release notes v.3.2, Modbus section.

helsmore commented 2 years ago

Hi @samson0v, I'm pretty sure that I had to rename the serial extension directory in /var/lib/thingsboard_gateway to myserial to solve the problem, restarting the service wasn't enough by itself to fix the problem, but I'll do a clean re-installation to confirm.

Also this doesn't solve the first part of the issue, where I need to rename the socket extension directory in /var/lib/thingsboard_gateway to mysocket to get the service to start at all after installation (as described in first post), that part of the problem occurred before I enabled the Modbus connector.

helsmore commented 2 years ago

You can see for example in the second set of logs the AttributeError: module 'serial' has no attribute 'serial_for_url' error indicating that Python has imported a copy of the serial module, but not the system one - I suspect it's imported the serial extension from /var/lib/thingsboard_gateway due to the service using that directory is its working directory.

helsmore commented 2 years ago

My apologies, it seems like I may have had items left in /var/lib/thingsboard_gateway from a previous installation that were causing the attribute error issues.

I used the following commands to cleanup and re-install:

sudo apt remove --purge python3-thingsboard-gateway
sudo rm -rf /var/log/thingsboard-gateway
sudo rm -rf /var/lib/thingsboard_gateway
sudo rm -rf /etc/thingsboard-gateway
sudo userdel -r thingsboard_gateway
sudo groupdel thingsboard_gateway
sudo reboot
sudo apt update && sudo apt full-upgrade
sudo apt install ./python3-thingsboard-gateway.deb -y
journalctl --follow -u thingsboard-gateway.service
sudo adduser thingsboard_gateway dialout
sudo mv /etc/thingsboard-gateway/config/tb_gateway.yaml /etc/thingsboard-gateway/config/tb_gateway.yaml.ORIG
sudo cp -a ~/tb_gateway.yaml.MOD /etc/thingsboard-gateway/config/tb_gateway.yaml
sudo cp -a ~/modbus_new_1.json.MOD /etc/thingsboard-gateway/config/modbus_new_1.json
sudo systemctl restart thingsboard-gateway.service
journalctl --follow -u thingsboard-gateway.service

After this cleanup and re-install the service was running fine and polling my Modbus RTU devices, no attribute errors in the logs.

Thanks for your help @samson0v, please also note that I didn't need to restart the service after the automatic installation of PyModbus and pySerial, it began working straight away.