Exception on read when no connection (self.transport is None).

PJ-Schulz commented 3 months ago

Versions

Python: 3.11.7
OS: Ubuntu 22.04 (wsl)
Pymodbus: 3.6.5
Modbus Hardware (if used):

Pymodbus Specific

Client: tcp - async

Description

Hello, i found a bug in the last release 3.6.5. You can see it with my example code. When I execute my code i get this Exception: TypeError: AsyncModbusTcpClient.close() got an unexpected keyword argument 'intern' when the client loses the connection to the server, for example when the server crashes. (here shown with docker kill)

Note: Without the asyncio.sleep the exception will be raised when i call client.connect.

Code and Logs

import asyncio
import subprocess

import pymodbus
from pymodbus.client import AsyncModbusTcpClient

pymodbus.pymodbus_apply_logging_config("DEBUG")

async def test_connection():
    subprocess.run(["docker", "run", "-d", "-p" "5020:5020", "--name=server", "oitc/modbus-server"])
    await asyncio.sleep(2)  # wait for container to start up

    client = AsyncModbusTcpClient("127.0.0.1", 5020)
    await client.connect()

    res = await client.read_holding_registers(100, 10)
    print(res)

    subprocess.run(["docker", "kill", "server"])
    subprocess.run(["docker", "rm", "server"])

    res = await client.read_holding_registers(100, 10)
    print(res)

asyncio.run(test_connection())

ad5068700d6897ea8b7d4015e8665db62eed314d25dce8b85f4a400b05852368
2024-03-08 09:08:50,701 DEBUG logging:103 Connecting to 127.0.0.1:5020.
2024-03-08 09:08:50,701 DEBUG logging:103 Connecting comm
2024-03-08 09:08:50,702 DEBUG logging:103 Connected to comm
2024-03-08 09:08:50,702 DEBUG logging:103 Adding transaction 1
2024-03-08 09:08:50,703 DEBUG logging:103 send: 0x0 0x1 0x0 0x0 0x0 0x6 0x0 0x3 0x0 0x64 0x0 0xa
2024-03-08 09:08:50,703 DEBUG logging:103 recv: 0x0 0x1 0x0 0x0 0x0 0x17 0x0 0x3 0x14 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 old_data:  addr=None
2024-03-08 09:08:50,703 DEBUG logging:103 Processing: 0x0 0x1 0x0 0x0 0x0 0x17 0x0 0x3 0x14 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
2024-03-08 09:08:50,704 DEBUG logging:103 Factory Response[ReadHoldingRegistersResponse': 3]
2024-03-08 09:08:50,704 DEBUG logging:103 Getting transaction 1
ReadHoldingRegistersResponse (10)
server
server
2024-03-08 09:08:51,261 DEBUG logging:103 Adding transaction 2
2024-03-08 09:08:51,261 DEBUG logging:103 send: 0x0 0x2 0x0 0x0 0x0 0x6 0x0 0x3 0x0 0x64 0x0 0xa
2024-03-08 09:08:51,261 DEBUG logging:103 -> transport: received eof
2024-03-08 09:08:51,261 DEBUG logging:103 Connection lost comm due to None
Exception in callback _SelectorSocketTransport._call_connection_lost(None)
handle: <Handle _SelectorSocketTransport._call_connection_lost(None)>
Traceback (most recent call last):
  File "/home/philipp/.pyenv/versions/3.11.7/lib/python3.11/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/philipp/.pyenv/versions/3.11.7/lib/python3.11/asyncio/selector_events.py", line 1117, in _call_connection_lost
    super()._call_connection_lost(exc)
  File "/home/philipp/.pyenv/versions/3.11.7/lib/python3.11/asyncio/selector_events.py", line 896, in _call_connection_lost
    self._protocol.connection_lost(exc)
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/.venv/lib/python3.11/site-packages/pymodbus/transport/transport.py", line 288, in connection_lost
    self.close(intern=True)
TypeError: AsyncModbusTcpClient.close() got an unexpected keyword argument 'intern'
2024-03-08 09:08:54,263 DEBUG logging:103 Adding transaction 2
2024-03-08 09:08:54,264 DEBUG logging:103 send: 0x0 0x2 0x0 0x0 0x0 0x6 0x0 0x3 0x0 0x64 0x0 0xa
2024-03-08 09:08:57,267 DEBUG logging:103 Adding transaction 2
2024-03-08 09:08:57,268 DEBUG logging:103 send: 0x0 0x2 0x0 0x0 0x0 0x6 0x0 0x3 0x0 0x64 0x0 0xa
2024-03-08 09:09:00,272 DEBUG logging:103 Adding transaction 2
2024-03-08 09:09:00,272 DEBUG logging:103 send: 0x0 0x2 0x0 0x0 0x0 0x6 0x0 0x3 0x0 0x64 0x0 0xa
2024-03-08 09:09:03,274 DEBUG logging:103 Connection lost comm due to Server not responding
Traceback (most recent call last):
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/foobar.py", line 26, in <module>
    asyncio.run(test_connection())
  File "/home/philipp/.pyenv/versions/3.11.7/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/home/philipp/.pyenv/versions/3.11.7/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/.pyenv/versions/3.11.7/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/foobar.py", line 23, in test_connection
    res = await client.read_holding_registers(100, 10)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/.venv/lib/python3.11/site-packages/pymodbus/client/base.py", line 179, in async_execute
    self.close(reconnect=True)
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/.venv/lib/python3.11/site-packages/pymodbus/client/tcp.py", line 92, in close
    super().close(reconnect=reconnect)
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/.venv/lib/python3.11/site-packages/pymodbus/client/base.py", line 130, in close
    self.connection_lost(asyncio.TimeoutError("Server not responding"))
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/.venv/lib/python3.11/site-packages/pymodbus/transport/transport.py", line 288, in connection_lost
    self.close(intern=True)
TypeError: AsyncModbusTcpClient.close() got an unexpected keyword argument 'intern'

janiversen commented 3 months ago

Thanks, we saw the same a couple of days ago, it is solved on dev. However we are right now pursuing a regression bug due to the fix.

janiversen commented 3 months ago

use dev + #2098 (which will be merged later today) solves the problem.

We are thinking of making a bug fix release.

janiversen commented 3 months ago

Solution available on dev, will be part of v3.6.6, which will be released later today.

PJ-Schulz commented 3 months ago

Thanks @janiversen. I updated to version 3.6.6 now this error does not occur any more.

However, I noticed something else with the same code, so i paste it here.

When I execute the same code, an AttributeError is raised, although I would expect a pymodbus.exceptions.ConnectionException.

2024-03-08 13:19:28,621 WARNING logging:109 Failed to connect [Errno 111] Connect call failed ('127.0.0.1', 5020)
2024-03-08 13:19:28,621 DEBUG logging:103 Wait comm 200.0 ms before reconnecting.
2024-03-08 13:19:28,821 DEBUG logging:103 Connecting to 127.0.0.1:5020.
2024-03-08 13:19:28,822 DEBUG logging:103 Connecting comm
2024-03-08 13:19:28,823 WARNING logging:109 Failed to connect [Errno 111] Connect call failed ('127.0.0.1', 5020)
2024-03-08 13:19:28,823 DEBUG logging:103 Wait comm 200.0 ms before reconnecting.
2024-03-08 13:19:28,896 DEBUG logging:103 send: 0x0 0x2 0x0 0x0 0x0 0x6 0x0 0x3 0x0 0x64 0x0 0xa
Traceback (most recent call last):
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/foobar.py", line 27, in <module>
    asyncio.run(test_connection())
  File "/home/philipp/.pyenv/versions/3.11.7/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/home/philipp/.pyenv/versions/3.11.7/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/.pyenv/versions/3.11.7/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/foobar.py", line 24, in test_connection
    res = await client.read_holding_registers(100, 10)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/.venv/lib/python3.11/site-packages/pymodbus/client/base.py", line 167, in async_execute
    self.send(packet)
  File "/home/philipp/repos/iiot-app-http-modbus-adapter/.venv/lib/python3.11/site-packages/pymodbus/transport/transport.py", line 391, in send
    self.transport.write(data)  # type: ignore[attr-defined]
    ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'write'
Future exception was never retrieved
future: <Future finished exception=ConnectionException('[Connection] Client is not connected')>
pymodbus.exceptions.ConnectionException: Modbus Error: [Connection] Client is not connected

Here I extend my code with catching the exceptions and the last lines of the output:

...
from pymodbus.exceptions import ConnectionException

async def test_connection():
    ...
    try:
        res = await client.read_holding_registers(100, 10)
        print(res)
    except AttributeError as ex:
        print("### AttributeError", ex)
    except ConnectionException as ex:
        print("### ConnectionException", ex)


2024-03-08 13:16:42,408 DEBUG logging:103 Connecting to 127.0.0.1:5020.
2024-03-08 13:16:42,409 DEBUG logging:103 Connecting comm
2024-03-08 13:16:42,410 WARNING logging:109 Failed to connect [Errno 111] Connect call failed ('127.0.0.1', 5020)
2024-03-08 13:16:42,410 DEBUG logging:103 Wait comm 200.0 ms before reconnecting.
2024-03-08 13:16:42,611 DEBUG logging:103 Connecting to 127.0.0.1:5020.
2024-03-08 13:16:42,612 DEBUG logging:103 Connecting comm
2024-03-08 13:16:42,613 WARNING logging:109 Failed to connect [Errno 111] Connect call failed ('127.0.0.1', 5020)
2024-03-08 13:16:42,613 DEBUG logging:103 Wait comm 200.0 ms before reconnecting.
2024-03-08 13:16:42,683 DEBUG logging:103 send: 0x0 0x2 0x0 0x0 0x0 0x6 0x0 0x3 0x0 0x64 0x0 0xa
### AttributeError 'NoneType' object has no attribute 'write'
Future exception was never retrieved
future: <Future finished exception=ConnectionException('[Connection] Client is not connected')>
pymodbus.exceptions.ConnectionException: Modbus Error: [Connection] Client is not connected

janiversen commented 3 months ago

Thanks for the info, seems you found an old bug, I thought we had solved long time ago....but it's probably reappeared due to the close() fix.

I will make a bug fix (but no new release) soon.

It looks to me as you do not test if the client is connected, before trying to read.

janiversen commented 3 months ago

Reopen due to new info.

pymodbus-dev / pymodbus