tjhowse / modbus4mqtt

Modbus TCP <-> MQTT glue. YAML configuration. Robust.
Other
78 stars 34 forks source link

Strange issues with version 0.5.0 #37

Closed plague-doctor closed 2 years ago

plague-doctor commented 2 years ago

Yesterday I have updated the version to 0.5.0 and it stopped working. It is throwing errors:

2021-10-10 23:20:52 INFO     Starting modbus4mqtt v0.5.0
2021-10-10 23:20:52 INFO     Connected to MQTT.
2021-10-10 23:20:52 ERROR    Failed to poll modbus device, attempting to reconnect: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll
    self._mb.poll()
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll
    values = self._scan_value_range(table, group, self._scan_batching)
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range
    result = self._mb.read_input_registers(start, count, unit=0x01)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers
    return self.execute(request)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute
    if not self.connect():
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect
    self._getkey()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 43, in _getkey
    self._setup()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 23, in _setup
    self._aes_ecb = AES.new(self._key, AES.MODE_ECB)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new
    return AESCipher(key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in __init__
    blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
    self._cipher = factory.new(key, *args, **kwargs)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
2021-10-10 23:21:52 ERROR    Failed to poll modbus device, attempting to reconnect: Modbus Error: [Connection] ModbusTcpClient(172.xxx.xxx.xxx:502): Connection unexpectedly closed 0.001899 seconds into read of 25 bytes without response from unit before it closed connection
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll
    self._mb.poll()
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll
    values = self._scan_value_range(table, group, self._scan_batching)
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range
    result = self._mb.read_input_registers(start, count, unit=0x01)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers
    return self.execute(request)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute
    if not self.connect():
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect
    self._getkey()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 40, in _getkey
    self._key_packet = self._recv(25)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 298, in _recv
    return self._handle_abrupt_socket_close(
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 343, in _handle_abrupt_socket_close
    raise ConnectionException(msg)
pymodbus.exceptions.ConnectionException: Modbus Error: [Connection] ModbusTcpClient(172.xxx.xxx.xxx:502): Connection unexpectedly closed 0.001899 seconds into read of 25 bytes without response from unit before it closed connection
2021-10-10 23:22:52 ERROR    Failed to poll modbus device, attempting to reconnect: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll
    self._mb.poll()
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll
    values = self._scan_value_range(table, group, self._scan_batching)
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range
    result = self._mb.read_input_registers(start, count, unit=0x01)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers
    return self.execute(request)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute
    if not self.connect():
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect
    self._getkey()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 43, in _getkey
    self._setup()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 23, in _setup
    self._aes_ecb = AES.new(self._key, AES.MODE_ECB)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new
    return AESCipher(key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in __init__
    blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
    self._cipher = factory.new(key, *args, **kwargs)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
2021-10-10 23:23:52 ERROR    Failed to poll modbus device, attempting to reconnect: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll
    self._mb.poll()
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll
    values = self._scan_value_range(table, group, self._scan_batching)
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range
    result = self._mb.read_input_registers(start, count, unit=0x01)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers
    return self.execute(request)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute
    if not self.connect():
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect
    self._getkey()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 43, in _getkey
    self._setup()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 23, in _setup
    self._aes_ecb = AES.new(self._key, AES.MODE_ECB)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new
    return AESCipher(key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in __init__
    blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
    self._cipher = factory.new(key, *args, **kwargs)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

When running directly from console, it is throwing errors but occasionally reads and publishes values. Once every 20-40 minutes. When running inside container I am still waiting for a single happy read - none so far for over an hour.

I call it like that:

modbus4mqtt --hostname mqtt.mydomain.net --port 1883 --username mqtt_user --password password123 --mqtt_topic_prefix solar --config ./SG5K-D.yaml

Please help, the previous version worked fine...

tjhowse commented 2 years ago

Hey! Thanks for raising this

It looks like an issue with pycrypto in Python 3.10. We have a dependency via SungrowModbusTcpClient.

An immediate stopgap fix for you would be to run with Python 3.9. I'll look at what would be involved in moving SungrowModbusTcpClient off pycrypto, which hasn't seen an update since 2013, and onto a newer, or built-in, crypto library.

Again, thanks for raising this issue! It's really helpful to find out when stuff breaks. I want modbus4mqtt to be useful to as many people as possible.

On Mon, 11 Oct 2021 at 09:31, Plague doctor @.***> wrote:

Yesterday I have updated the version to 0.5.0 and it stopped working. It is throwing errors:

2021-10-10 23:20:52 INFO Starting modbus4mqtt v0.5.0 2021-10-10 23:20:52 INFO Connected to MQTT. 2021-10-10 23:20:52 ERROR Failed to poll modbus device, attempting to reconnect: PY_SSIZE_T_CLEAN macro must be defined for '#' formats Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll self._mb.poll() File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll values = self._scan_value_range(table, group, self._scan_batching) File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range result = self._mb.read_input_registers(start, count, unit=0x01) File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers return self.execute(request) File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute if not self.connect(): File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect self._getkey() File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 43, in _getkey self._setup() File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 23, in _setup self._aes_ecb = AES.new(self._key, AES.MODE_ECB) File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new return AESCipher(key, *args, kwargs) File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in init blockalgo.BlockAlgo.init(self, _AES, key, *args, *kwargs) File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in init self._cipher = factory.new(key, args, kwargs) SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats 2021-10-10 23:21:52 ERROR Failed to poll modbus device, attempting to reconnect: Modbus Error: [Connection] ModbusTcpClient(172.xxx.xxx.xxx:502): Connection unexpectedly closed 0.001899 seconds into read of 25 bytes without response from unit before it closed connection Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll self._mb.poll() File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll values = self._scan_value_range(table, group, self._scan_batching) File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range result = self._mb.read_input_registers(start, count, unit=0x01) File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers return self.execute(request) File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute if not self.connect(): File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect self._getkey() File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 40, in _getkey self._key_packet = self._recv(25) File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 298, in _recv return self._handle_abrupt_socket_close( File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 343, in _handle_abrupt_socket_close raise ConnectionException(msg) pymodbus.exceptions.ConnectionException: Modbus Error: [Connection] ModbusTcpClient(172.xxx.xxx.xxx:502): Connection unexpectedly closed 0.001899 seconds into read of 25 bytes without response from unit before it closed connection 2021-10-10 23:22:52 ERROR Failed to poll modbus device, attempting to reconnect: PY_SSIZE_T_CLEAN macro must be defined for '#' formats Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll self._mb.poll() File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll values = self._scan_value_range(table, group, self._scan_batching) File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range result = self._mb.read_input_registers(start, count, unit=0x01) File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers return self.execute(request) File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute if not self.connect(): File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect self._getkey() File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 43, in _getkey self._setup() File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 23, in _setup self._aes_ecb = AES.new(self._key, AES.MODE_ECB) File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new return AESCipher(key, *args, kwargs) File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in init blockalgo.BlockAlgo.init(self, _AES, key, *args, *kwargs) File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in init self._cipher = factory.new(key, args, kwargs) SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats 2021-10-10 23:23:52 ERROR Failed to poll modbus device, attempting to reconnect: PY_SSIZE_T_CLEAN macro must be defined for '#' formats Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll self._mb.poll() File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll values = self._scan_value_range(table, group, self._scan_batching) File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range result = self._mb.read_input_registers(start, count, unit=0x01) File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers return self.execute(request) File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute if not self.connect(): File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect self._getkey() File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 43, in _getkey self._setup() File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 23, in _setup self._aes_ecb = AES.new(self._key, AES.MODE_ECB) File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new return AESCipher(key, *args, kwargs) File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in init blockalgo.BlockAlgo.init(self, _AES, key, *args, *kwargs) File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in init self._cipher = factory.new(key, args, kwargs) SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

When running directly from console, it is throwing errors but occasionally reads and publishes values. Once every 20-40 minutes. When running inside container I am still waiting for a single happy read - none so far for over an hour.

I call it like that:

modbus4mqtt --hostname mqtt.mydomain.net --port 1883 --username mqtt_user --password password123 --mqtt_topic_prefix solar --config ./SG5K-D.yaml

Please help, the previous version worked fine...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tjhowse/modbus4mqtt/issues/37, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHWDUN7HJ3XEEVS7EDH77LUGIO4TANCNFSM5FW7WPNA .

tjhowse commented 2 years ago

This should be fixed with v0.5.1. Please update and let me know how you go.

plague-doctor commented 2 years ago

Works again. Thanks :-)