modbus integration is blocked by a non-responsive sensor

nrodday commented 1 month ago

The problem

When using multiple modbus rtu entities a single non-responsive sensor blocks all other modbus communication on that channel. Therefore additional modules, like relays, won´t work anymore when a sensor gets disconnected.

What version of Home Assistant Core has the issue?

2024.7.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Core

Integration causing the issue

modbus

Link to integration documentation on our website

https://www.home-assistant.io/integrations/modbus/

Diagnostics information

No response

Example YAML snippet

switches:
      - name: Relay1
        device_address: 1
        address: 0
        write_type: coil
        command_on: 1
        command_off: 0
      - name: Relay2
        device_address: 1
        address: 1
        write_type: coil
        command_on: 1
        command_off: 0
      - name: Relay3
        device_address: 1
        address: 2
        write_type: coil
        command_on: 1
        command_off: 0
      - name: Relay4
        device_address: 1
        address: 3
        write_type: coil
        command_on: 1
        command_off: 0

    sensors:
      - name: PH Sensor Temp
        slave: 12
        address: 0
        input_type: holding
        data_type: int16
        unit_of_measurement: '°C'
        scale: 0.01
        precision: 2
      - name: PH Sensor PH
        slave: 12
        address: 1
        input_type: holding
        data_type: uint16
        scale: 0.01
        precision: 2
        device_class: ph
        min_value: 0
        max_value: 14

      - name: TH04S Humidity
        slave: 21
        address: 0
        input_type: holding
        data_type: uint16

      - name: TH04S Temperature
        slave: 21
        address: 1
        input_type: holding
        data_type: int16

Anything in the logs that might be useful for us?

This is how it look when all sensors are connected:

2024-07-25 19:41:56.067 DEBUG (MainThread) [pymodbus.logging] Getting transaction 12
2024-07-25 19:41:56.099 DEBUG (MainThread) [pymodbus.logging] Adding transaction 21
2024-07-25 19:41:56.099 DEBUG (MainThread) [pymodbus.logging] Resetting frame - Current Frame in buffer -
2024-07-25 19:41:56.099 DEBUG (MainThread) [pymodbus.logging] send: 0x15 0x3 0x0 0x0 0x0 0x1 0x87 0x1e
2024-07-25 19:41:56.123 DEBUG (MainThread) [pymodbus.logging] recv: 0x15 0x3 0x2 0x1 0xfe 0x8 0x57 old_data:  addr=None
2024-07-25 19:41:56.123 DEBUG (MainThread) [pymodbus.logging] Processing: 0x15 0x3 0x2 0x1 0xfe 0x8 0x57
2024-07-25 19:41:56.124 DEBUG (MainThread) [pymodbus.logging] Getting Frame - 0x3 0x2 0x1 0xfe
2024-07-25 19:41:56.124 DEBUG (MainThread) [pymodbus.logging] Factory Response[ReadHoldingRegistersResponse': 3]
2024-07-25 19:41:56.124 DEBUG (MainThread) [pymodbus.logging] Frame advanced, resetting header!!
2024-07-25 19:41:56.124 DEBUG (MainThread) [pymodbus.logging] Getting transaction 21
2024-07-25 19:41:56.156 DEBUG (MainThread) [pymodbus.logging] Adding transaction 21
2024-07-25 19:41:56.156 DEBUG (MainThread) [pymodbus.logging] Resetting frame - Current Frame in buffer -
2024-07-25 19:41:56.156 DEBUG (MainThread) [pymodbus.logging] send: 0x15 0x3 0x0 0x1 0x0 0x1 0xd6 0xde
2024-07-25 19:41:56.181 DEBUG (MainThread) [pymodbus.logging] recv: 0x15 0x3 0x2 0x1 0x4 0x88 0x14 old_data:  addr=None
2024-07-25 19:41:56.181 DEBUG (MainThread) [pymodbus.logging] Processing: 0x15 0x3 0x2 0x1 0x4 0x88 0x14
2024-07-25 19:41:56.181 DEBUG (MainThread) [pymodbus.logging] Getting Frame - 0x3 0x2 0x1 0x4
2024-07-25 19:41:56.182 DEBUG (MainThread) [pymodbus.logging] Factory Response[ReadHoldingRegistersResponse': 3]
2024-07-25 19:41:56.182 DEBUG (MainThread) [pymodbus.logging] Frame advanced, resetting header!!
2024-07-25 19:41:56.182 DEBUG (MainThread) [pymodbus.logging] Getting transaction 21

And this is how it looks when I disconnected a sensor:

2024-07-25 19:42:41.185 DEBUG (MainThread) [pymodbus.logging] Frame advanced, resetting header!!
2024-07-25 19:42:41.186 DEBUG (MainThread) [pymodbus.logging] Getting transaction 21
2024-07-25 19:42:55.990 DEBUG (MainThread) [pymodbus.logging] Adding transaction 12
2024-07-25 19:42:55.990 DEBUG (MainThread) [pymodbus.logging] Resetting frame - Current Frame in buffer -
2024-07-25 19:42:55.991 DEBUG (MainThread) [pymodbus.logging] send: 0xc 0x3 0x0 0x0 0x0 0x1 0x85 0x17
2024-07-25 19:43:00.992 DEBUG (MainThread) [pymodbus.logging] Adding transaction 12
2024-07-25 19:43:00.993 DEBUG (MainThread) [pymodbus.logging] Resetting frame - Current Frame in buffer -
2024-07-25 19:43:00.993 DEBUG (MainThread) [pymodbus.logging] send: 0xc 0x3 0x0 0x0 0x0 0x1 0x85 0x17
2024-07-25 19:43:05.994 DEBUG (MainThread) [pymodbus.logging] Adding transaction 12
2024-07-25 19:43:05.995 DEBUG (MainThread) [pymodbus.logging] Resetting frame - Current Frame in buffer -
2024-07-25 19:43:05.995 DEBUG (MainThread) [pymodbus.logging] send: 0xc 0x3 0x0 0x0 0x0 0x1 0x85 0x17
2024-07-25 19:43:10.996 DEBUG (MainThread) [pymodbus.logging] Adding transaction 12
2024-07-25 19:43:10.997 DEBUG (MainThread) [pymodbus.logging] Resetting frame - Current Frame in buffer -
2024-07-25 19:43:10.997 DEBUG (MainThread) [pymodbus.logging] send: 0xc 0x3 0x0 0x0 0x0 0x1 0x85 0x17
2024-07-25 19:43:15.999 DEBUG (MainThread) [pymodbus.logging] Connection lost comm due to Server not responding
2024-07-25 19:43:16.000 DEBUG (MainThread) [pymodbus.logging] callback_disconnected called: Server not responding
2024-07-25 19:43:16.001 ERROR (MainThread) [homeassistant.components.modbus.modbus] Pymodbus: modbus_hub: Error: device: 12 address: 0 -> Modbus Error: [Input/Output] ERROR: No response received after 3 retries
2024-07-25 19:43:16.001 DEBUG (MainThread) [pymodbus.logging] Wait comm 100.0 ms before reconnecting.
2024-07-25 19:43:16.033 DEBUG (MainThread) [homeassistant.components.modbus.modbus] Pymodbus: modbus_hub: Error: device: 12 address: 1 -> Modbus Error: [Connection] Not connected[AsyncModbusSerialClient /dev/ttyAMA5:0]
2024-07-25 19:43:16.065 DEBUG (MainThread) [homeassistant.components.modbus.modbus] Pymodbus: modbus_hub: Error: device: 21 address: 0 -> Modbus Error: [Connection] Not connected[AsyncModbusSerialClient /dev/ttyAMA5:0]
2024-07-25 19:43:16.098 DEBUG (MainThread) [homeassistant.components.modbus.modbus] Pymodbus: modbus_hub: Error: device: 21 address: 1 -> Modbus Error: [Connection] Not connected[AsyncModbusSerialClient /dev/ttyAMA5:0]
2024-07-25 19:43:16.102 DEBUG (MainThread) [pymodbus.logging] Connecting to /dev/ttyAMA5.
2024-07-25 19:43:16.102 DEBUG (MainThread) [pymodbus.logging] Connecting comm
2024-07-25 19:43:16.105 DEBUG (MainThread) [pymodbus.logging] Connected to comm
2024-07-25 19:43:16.105 DEBUG (MainThread) [pymodbus.logging] Resetting frame - Current Frame in buffer -
2024-07-25 19:43:16.129 DEBUG (MainThread) [pymodbus.logging] Adding transaction 1

Additional information

When a relay is triggered during the nonresponsiveness of a sensor nothing happens, as the command is not transmitted. It is only switched on after the timeout message "Server not responding". Milliseconds after, the integration again enters into the blocking state while waiting for a reply and nothing does through anymore.

It seems the communication on the channel is always blocking until an answer to every sent packet is received.

How is it possible to parallelize the communication with multiple sensors?

home-assistant[bot] commented 1 month ago

modbus documentation modbus source

nrodday commented 1 month ago

I have now upgraded to HA 2024.7.3 and frontend 20240710.0. Moreover, I have upgraded to the latest pymodbus package 3.6.9.

The issue still persist. When a single sensor is disconnected, the integration just repeats the send command for the unresponsive sensor but does not send out any other modbus communication anymore.

nrodday commented 1 month ago

@janiversen Since you are the maintainer of the modbus integration, I guess that this issue is for you? If it is not a pymodbus problem, it certainly is a HA modbus integration problem.

janiversen commented 1 month ago

I am not the maintainer of the modbus integration any longer, please do not address me directly.

nrodday commented 1 month ago

Anyone else having an idea on how to turn off the resetting of frames (and therefore repeating them) when a slave does not respond? This might resolve the blocking of the channel during the wait periods. The desired behavior should simply be a log entry and then moving forward with the next command in the queue. Setting "timeout = 0.1" reduces the blocked periods significantly, but isn´t a production solution.

Moreover, I suspect a bug in this implementation as the disconnect of one sensor should not impact the querying of other sensors.

home-assistant / core