nathanmarlor / foxess_modbus

FoxESS inverter integration. Connect directly to your FoxESS inverter (no cloud!) for real-time status and control.
MIT License
110 stars 15 forks source link

Serial connections fail when there are many open fds #457

Closed djdarlo closed 8 months ago

djdarlo commented 10 months ago

Version of the custom_component

Configuration

Add your logs here.

Describe the bug

A clear and concise description of what the bug is.

I am using the modbus serial integration. Every so often the sensors stop reporting. A simple restart of HA sorts it but I have noticed these errors in the log.

Debug log


Add your logs here.

This error originated from a custom integration.

Logger: custom_components.foxess_modbus.modbus_controller
Source: custom_components/foxess_modbus/modbus_controller.py:220
Integration: FoxESS - Modbus (documentation, issues)
First occurred: 06:35:45 (1 occurrences)
Last logged: 06:35:45

/dev/ttyUSB0 247 - 5 failed poll attempts: now not connected. Last error: filedescriptor out of range in select()

This error originated from a custom integration.

Logger: custom_components.foxess_modbus.modbus_controller
Source: custom_components/foxess_modbus/modbus_controller.py:152
Integration: FoxESS - Modbus (documentation, issues)
First occurred: 06:34:57 (800 occurrences)
Last logged: 09:54:46

General exception when polling /dev/ttyUSB0 247: ValueError('filedescriptor out of range in select()')
Traceback (most recent call last):
  File "/config/custom_components/foxess_modbus/modbus_controller.py", line 152, in _refresh
    reads = await self._client.read_registers(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/foxess_modbus/modbus_client.py", line 210, in read_registers
    response = await self._async_pymodbus_call(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/foxess_modbus/modbus_client.py", line 298, in _async_pymodbus_call
    result = await self._hass.async_add_executor_job(call, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/client/mixin.py", line 116, in read_input_registers
    return self.execute(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/client/base.py", line 192, in execute
    return self.transaction.execute(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/transaction.py", line 159, in execute
    response, last_exception = self._transact(
                               ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/transaction.py", line 275, in _transact
    size = self._send(packet)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/transaction.py", line 323, in _send
    return self.client.framer.sendPacket(packet)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/framer/rtu_framer.py", line 296, in sendPacket
    size = self.client.send(message)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/client/serial.py", line 278, in send
    size = self.socket.write(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/serial/serialposix.py", line 640, in write
    abort, ready, _ = select.select([self.pipe_abort_write_r], [self.fd], [], None)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: filedescriptor out of range in select()
canton7 commented 10 months ago

Looks the same as #273, but for USB devices rather than network devices.

The problem is that you have too many file descriptors open (more than 1024), and Python's serial library can't cope.

I'm not honestly sure what we're going to be able to do about it, as the problematic code is two layers of abstraction below us.

canton7 commented 10 months ago

Hmm, looks like pyserial has a PosixPollSerial class, which will do what we want. Trying to work out how to get pymodbus to use it...

canton7 commented 10 months ago

https://pyserial.readthedocs.io/en/latest/url_handlers.html#alt. I'll try and give that a try over the weekend, although I'll need some help testing as I don't have a serial USB device to test with.

djdarlo commented 10 months ago

Happy to give it a test for you. I’ve got a couple of HA setups using this.

On Fri, 3 Nov 2023 at 10:07, Antony Male @.***> wrote:

https://pyserial.readthedocs.io/en/latest/url_handlers.html#alt. I'll try and give that a try over the weekend, although I'll need some help testing as I don't have a serial USB device to test with.

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1792163545, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2D7QVJJM7PZM5MXKJTYCS65PAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJSGE3DGNJUGU . You are receiving this because you authored the thread.Message ID: @.***>

canton7 commented 10 months ago

Reopening until we've confirmed this works

canton7 commented 10 months ago

@djdarlo Would you mind trying out the pre-release v1.8.0b3, see if that fixes it for you? Thanks!

djdarlo commented 10 months ago

Of course. Will try it later today.

On Sat, 4 Nov 2023 at 12:15, Antony Male @.***> wrote:

@djdarlo https://github.com/djdarlo Would you mind trying out the pre-release v1.8.0b3, see if that fixes it for you? Thanks!

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1793427382, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2FJLZTVHEQBI6HEAEDYCYWVHAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJTGQZDOMZYGI . You are receiving this because you were mentioned.Message ID: @.***>

djdarlo commented 10 months ago

How do I download a pre release?

On Sat, 4 Nov 2023 at 12:18, David Darlington @.***> wrote:

Of course. Will try it later today.

On Sat, 4 Nov 2023 at 12:15, Antony Male @.***> wrote:

@djdarlo https://github.com/djdarlo Would you mind trying out the pre-release v1.8.0b3, see if that fixes it for you? Thanks!

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1793427382, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2FJLZTVHEQBI6HEAEDYCYWVHAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJTGQZDOMZYGI . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 10 months ago

Hacs - Integrations - FoxESS Modbus - Hamburger menu in the top right - Redownload, then select the "Show beta versions" toggle

djdarlo commented 10 months ago

Thank you. I’ve upgraded one and left one as is. Working. I’ll keep you updated on how it goes. Thanks again.

On Sat, 4 Nov 2023 at 14:16, Antony Male @.***> wrote:

Hacs - Integrations - FoxESS Modbus - Hamburger menu in the top right - Redownload, then select the "Show beta versions" toggle

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1793455058, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2AZKVV5II7HDOKXEQTYCZE2NAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJTGQ2TKMBVHA . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 10 months ago

Thanks!

djdarlo commented 10 months ago

The other one errored yesterday so I've updated that one too. Both going fine so far. I'll keep you updated. Thank you.

djdarlo commented 10 months ago

I spoke too soon. One of my HA installs went pop over night :( Other one is still going fine. Logs below...

1)

This error originated from a custom integration.

Logger: custom_components.foxess_modbus.modbus_controller
Source: custom_components/foxess_modbus/modbus_controller.py:129
Integration: FoxESS - Modbus (documentation, issues)
First occurred: November 5, 2023 at 8:30:16 PM (1 occurrences)
Last logged: November 5, 2023 at 8:30:16 PM

Aborting refresh of /dev/ttyUSB0 247 as a previous refresh is still in progress. Is your poll rate '15' too high?

2)

This error originated from a custom integration.

Logger: custom_components.foxess_modbus.modbus_controller
Source: custom_components/foxess_modbus/modbus_controller.py:153
Integration: FoxESS - Modbus (documentation, issues)
First occurred: 1:45:24 AM (3 occurrences)
Last logged: 1:47:24 AM

General exception when polling /dev/ttyUSB0 247: UnboundLocalError("cannot access local variable 'fd' where it is not associated with a value")
Traceback (most recent call last):
  File "/config/custom_components/foxess_modbus/modbus_controller.py", line 153, in _refresh
    reads = await self._client.read_registers(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/foxess_modbus/modbus_client.py", line 218, in read_registers
    response = await self._async_pymodbus_call(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/foxess_modbus/modbus_client.py", line 306, in _async_pymodbus_call
    result = await self._hass.async_add_executor_job(call, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/client/mixin.py", line 117, in read_input_registers
    pdu_reg_read.ReadInputRegistersRequest(address, count, slave, **kwargs)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/client/base.py", line 177, in execute
    self.delay_ms = self.params.reconnect_delay
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/transaction.py", line 168, in execute
    )

  File "/usr/local/lib/python3.11/site-packages/pymodbus/transaction.py", line 313, in _transact
    ) as msg:

  File "/usr/local/lib/python3.11/site-packages/pymodbus/transaction.py", line 392, in _recv
    result = read_min + result
             ~~~~~~~~~^~~~~~~~~
  File "/usr/local/lib/python3.11/site-packages/pymodbus/framer/rtu_framer.py", line 296, in recvPacket
    size = self.client.send(message)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/client/serial.py", line 271, in recv
    return result
  File "/usr/local/lib/python3.11/site-packages/serial/serialposix.py", line 830, in read
    if fd == self.pipe_abort_read_r:
       ^^
UnboundLocalError: cannot access local variable 'fd' where it is not associated with a value

3)

This error originated from a custom integration.

Logger: custom_components.foxess_modbus.modbus_controller
Source: custom_components/foxess_modbus/modbus_controller.py:221
Integration: FoxESS - Modbus (documentation, issues)
First occurred: 1:50:23 AM (1 occurrences)
Last logged: 1:50:23 AM

/dev/ttyUSB0 247 - 5 failed poll attempts: now not connected. Last error: filedescriptor out of range in select()

4)

Logger: pymodbus.logging
Source: /usr/local/lib/python3.11/site-packages/pymodbus/logging.py:108
First occurred: 1:45:21 AM (1181 occurrences)
Last logged: 6:44:01 AM

Cleanup recv buffer before send: 0xf7 0x4 0xba 0x5 0x88 0x0 0x0 0x0 0x4 0x3 0xb9 0x0 0x0 0xff 0xfe 0x9 0x38 0x0 0x17 0x2 0x2a 0x9 0x72 0x0 0x17 0x2 0x14 0xff 0xd7 0x2 0x34 0x13 0x92 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xf7 0x0 0x0 0x2 0x21 0x1 0x1a 0x1 0x78 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x35 0xff 0xe8 0x0 0x2f 0x2 0x28 0x0 0xe5 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x0 0xf1 0x0 0xb9 0xc 0xc7 0xc 0xb9 0x4 0xf 0x0 0xb2 0xe0 0xb 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xcc 0x58 0x0 0x0 0x0 0x0 0x4d 0xad 0x0 0x1c 0x0 0x0 0x48 0x6a 0x0 0x10 0x0 0x0 0x4b 0xd6 0x0 0x0 0x0 0x0 0x31 0xde 0x0 0x1f 0x0 0x0 0xcd 0x53 0x0 0xe 0x0 0x0 0x19 0xb1 0x0 0x1e 0x0 0x0 0x97 0x82 0x0 0xf 0x95 0x95
Cleanup recv buffer before send: 0xf7 0x4 0xba 0x5 0x79 0x0 0x0 0x0 0x5 0x3 0xc3 0x0 0x0 0xff 0xff 0x9 0x38 0x0 0x17 0x2 0x2c 0x9 0x73 0x0 0x17 0x2 0x11 0xff 0xd9 0x2 0x32 0x13 0x90 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xee 0x0 0x0 0x2 0x23 0x1 0x1a 0x1 0x78 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x35 0xff 0xe7 0x0 0x2f 0x2 0x27 0x0 0xe5 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x0 0xf1 0x0 0xb9 0xc 0xc6 0xc 0xb9 0x4 0xf 0x0 0xb2 0xe0 0xd 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xcc 0x58 0x0 0x0 0x0 0x0 0x4d 0xad 0x0 0x1c 0x0 0x0 0x48 0x6a 0x0 0x10 0x0 0x0 0x4b 0xd6 0x0 0x0 0x0 0x0 0x31 0xde 0x0 0x1f 0x0 0x0 0xcd 0x53 0x0 0xe 0x0 0x0 0x19 0xb1 0x0 0x1e 0x0 0x0 0x97 0x82 0x0 0xf 0x8e 0x9a
Cleanup recv buffer before send: 0xf7 0x4 0xba 0x5 0x9c 0x0 0x0 0x0 0x4 0x3 0xcc 0x0 0x0 0xff 0xfd 0x9 0x38 0x0 0x17 0x2 0x27 0x9 0x73 0x0 0x17 0x2 0x10 0xff 0xd7 0x2 0x31 0x13 0x8d 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xfe 0x0 0x0 0x2 0x12 0x1 0x1a 0x1 0x78 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x35 0xff 0xe8 0x0 0x2f 0x2 0x27 0x0 0xe5 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x0 0xf1 0x0 0xb8 0xc 0xc6 0xc 0xba 0x4 0xf 0x0 0xb2 0xe0 0x10 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xcc 0x58 0x0 0x0 0x0 0x0 0x4d 0xad 0x0 0x1c 0x0 0x0 0x48 0x6a 0x0 0x10 0x0 0x0 0x4b 0xd6 0x0 0x0 0x0 0x0 0x31 0xde 0x0 0x1f 0x0 0x0 0xcd 0x53 0x0 0xe 0x0 0x0 0x19 0xb1 0x0 0x1e 0x0 0x0 0x97 0x82 0x0 0xf 0xc6 0x9b
Cleanup recv buffer before send: 0xf7 0x4 0xba 0x5 0x9f 0x0 0x0 0x0 0x4 0x3 0xd4 0x0 0x0 0x0 0x0 0x9 0x38 0x0 0x17 0x2 0x27 0x9 0x6c 0x0 0x17 0x2 0xc 0xff 0xdd 0x2 0x2c 0x13 0x8e 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xf1 0x0 0x0 0x2 0x1c 0x1 0x1a 0x1 0x78 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x35 0xff 0xe8 0x0 0x2f 0x2 0x27 0x0 0xe5 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x0 0xf1 0x0 0xb8 0xc 0xc6 0xc 0xba 0x4 0xf 0x0 0xb2 0xe0 0x12 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xcc 0x58 0x0 0x0 0x0 0x0 0x4d 0xad 0x0 0x1c 0x0 0x0 0x48 0x6a 0x0 0x10 0x0 0x0 0x4b 0xd6 0x0 0x0 0x0 0x0 0x31 0xde 0x0 0x1f 0x0 0x0 0xcd 0x53 0x0 0xe 0x0 0x0 0x19 0xb1 0x0 0x1e 0x0 0x0 0x97 0x82 0x0 0xf 0x2 0xa1
Cleanup recv buffer before send: 0xf7 0x4 0xba 0x5 0x66 0x0 0x0 0x0 0x7 0x3 0xdd 0x0 0x0 0xff 0xfd 0x9 0x38 0x0 0x17 0x2 0x2f 0x9 0x6f 0x0 0x17 0x2 0x17 0xff 0xd9 0x2 0x38 0x13 0x8f 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xed 0x0 0x0 0x2 0x2b 0x1 0x1a 0x1 0x78 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x37 0xff 0xe8 0x0 0x2f 0x2 0x27 0x0 0xe5 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x0 0xf1 0x0 0xb8 0xc 0xc6 0xc 0xba 0x4 0xf 0x0 0xb2 0xe0 0x15 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xcc 0x58 0x0 0x0 0x0 0x0 0x4d 0xad 0x0 0x1c 0x0 0x0 0x48 0x6a 0x0 0x10 0x0 0x0 0x4b 0xd6 0x0 0x0 0x0 0x0 0x31 0xde 0x0 0x1f 0x0 0x0 0xcd 0x53 0x0 0xe 0x0 0x0 0x19 0xb1 0x0 0x1e 0x0 0x0 0x97 0x82 0x0 0xf 0xd4 0x21

5)

This error originated from a custom integration.

Logger: custom_components.foxess_modbus.modbus_controller
Source: custom_components/foxess_modbus/modbus_controller.py:153
Integration: FoxESS - Modbus (documentation, issues)
First occurred: 1:49:35 AM (1178 occurrences)
Last logged: 6:44:01 AM

General exception when polling /dev/ttyUSB0 247: ValueError('filedescriptor out of range in select()')
Traceback (most recent call last):
  File "/config/custom_components/foxess_modbus/modbus_controller.py", line 153, in _refresh
    reads = await self._client.read_registers(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/foxess_modbus/modbus_client.py", line 218, in read_registers
    response = await self._async_pymodbus_call(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/foxess_modbus/modbus_client.py", line 306, in _async_pymodbus_call
    result = await self._hass.async_add_executor_job(call, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/client/mixin.py", line 117, in read_input_registers
    pdu_reg_read.ReadInputRegistersRequest(address, count, slave, **kwargs)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/client/base.py", line 177, in execute
    self.delay_ms = self.params.reconnect_delay
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/transaction.py", line 168, in execute
    )

  File "/usr/local/lib/python3.11/site-packages/pymodbus/transaction.py", line 285, in _transact
    return size, None
       ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/transaction.py", line 325, in _send
    def _recv(self, expected_response_length, full):  # pylint: disable=too-complex
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymodbus/framer/rtu_framer.py", line 286, in sendPacket
    break
  File "/usr/local/lib/python3.11/site-packages/pymodbus/client/serial.py", line 231, in send
    self.last_frame_end = None
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/serial/serialposix.py", line 640, in write
    abort, ready, _ = select.select([self.pipe_abort_write_r], [self.fd], [], None)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: filedescriptor out of range in select()

home-assistant_2023-11-07T06-48-03.419Z.zip

canton7 commented 10 months ago

Are you sure you're running v1.8.0b3? The line numbers in that stack trace line up with v1.7.6, but not v1.8.0b3.

Remember you need to restart HA (not just a "quick reload") after updating/installing custom integrations.

djdarlo commented 10 months ago

I think so....

[image: image.png]

On Tue, 7 Nov 2023 at 10:47, Antony Male @.***> wrote:

Are you sure you're running v1.8.0b3? The line numbers in that stack trace line up with v1.7.6, but not v1.8.0b3.

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1798259893, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2DED6NU5ZA7W4HDG2LYDIGUPAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJYGI2TSOBZGM . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 10 months ago

You need to go to the github web interface to upload images

djdarlo commented 10 months ago

Apologies... :) modbus

canton7 commented 10 months ago

Aha, no, that's my mistake.

Slightly confused by what's going on in pyserial there. I'll have a deeper look when I get the chance

canton7 commented 9 months ago

Sorry for the delay!

Looks like this is a known issue in pyserial: https://github.com/pyserial/pyserial/issues/617.

I've had a go at working around the issue. Can you give v1.8.0b5 a try please?

djdarlo commented 9 months ago

Of course. Will put it one of my systems now. I’d currently got a daily restart in place to work around it so will remove that too.

D

On Wed, 29 Nov 2023 at 10:28, Antony Male @.***> wrote:

Sorry for the delay!

Looks like this is a known issue in pyserial: pyserial/pyserial#617 https://github.com/pyserial/pyserial/issues/617.

I've had a go at working around the issue. Can you give v1.8.0b5 a try please?

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1831622926, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2CWPSU3PX4XVAXZMX3YG4E6DAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZRGYZDEOJSGY . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 9 months ago

Thanks!

canton7 commented 9 months ago

Hi, any news on whether it's behaving better?

djdarlo commented 9 months ago

Hey there. It seemed to fail again so I have reverted to a daily restart. I’ll capture the logs next time and send them over to you.

On Sat, 9 Dec 2023 at 11:27, Antony Male @.***> wrote:

Hi, any news on whether it's behaving better?

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1848380497, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2DJ3TI24WKBE7MP55TYIRDLJAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBYGM4DANBZG4 . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 9 months ago

Thanks! It's likely a different problem to last time. I'm afraid I don't have a serial devie here, so I'm having to develop this workaround by proxy.

djdarlo commented 9 months ago

Not a problem - its just gone pop again - here you go :)

Home Assistant Core

*Connection to (192.168.1.102, 502) failed: [Errno 113] Host is unreachable3:32:36 PM – (ERROR) components/modbus/modbus.py - message first occurred at December 10, 2023 at 11:19:32 PM and shows up 37187 timesGeneral exception when polling /dev/ttyUSB0 247: ValueError('filedescriptor out of range in select()')3:32:34 PM – (WARNING) FoxESS - Modbus (custom integration) - message first occurred at 3:30:21 PM and shows up 14 timesCleanup recv buffer before send: 0xf7 0x4 0xba 0xc 0x81 0x0 0x1 0x0 0x24 0x5 0x5f 0x0 0x0 0x0 0x3 0x9 0x5d 0x0 0x13 0x1 0xd0 0x9 0x5d 0x0 0x15 0x1 0xd9 0xff 0xdf 0x1 0xfc 0x13 0x7f 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xf8 0x0 0x0 0x1 0xe6 0x1 0x52 0x1 0xb1 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x5a 0xff 0xee 0x0 0x4e 0x3 0x8a 0x1 0xe 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x1 0x1f 0x0 0xe3 0xc 0xf7 0xc 0xeb 0x0 0x4 0x0 0xd1 0x79 0x95 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xd1 0xde 0x0 0x2e 0x0 0x0 0x5d 0x49 0x0 0x64 0x0 0x0 0x56 0xf2 0x0 0x31 0x0 0x0 0x4d 0x8d 0x0 0x1 0x0 0x0 0x44 0x16 0x0 0x5a 0x0 0x0 0xde 0x2e 0x0 0x48 0x0 0x0 0x28 0x49 0x0 0x58 0x0 0x0 0xaa 0x42 0x0 0x4a 0x81 0xaf3:32:34 PM – (WARNING) /usr/local/lib/python3.11/site-packages/pymodbus/logging.py

Cleanup recv buffer before send: 0xf7 0x4 0xba 0xc 0x81 0x0 0x1 0x0 0x24 0x5 0x5f 0x0 0x0 0x0 0x3 0x9 0x5d 0x0 0x13 0x1 0xd0 0x9 0x5d 0x0 0x15 0x1 0xd9 0xff 0xdf 0x1 0xfc 0x13 0x7f 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xf8 0x0 0x0 0x1 0xe6 0x1 0x52 0x1 0xb1 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x5a 0xff 0xee 0x0 0x4e 0x3 0x8a 0x1 0xe 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x1 0x1f 0x0 0xe3 0xc 0xf7 0xc 0xeb 0x0 0x4 0x0 0xd1 0x79 0x95 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xd1 0xde 0x0 0x2e 0x0 0x0 0x5d 0x49 0x0 0x64 0x0 0x0 0x56 0xf2 0x0 0x31 0x0 0x0 0x4d 0x8d 0x0 0x1 0x0 0x0 0x44 0x16 0x0 0x5a 0x0 0x0 0xde 0x2e 0x0 0x48 0x0 0x0 0x28 0x49 0x0 0x58 0x0 0x0 0xaa 0x42 0x0 0x4a 0x81 0xaf Cleanup recv buffer before send: 0xf7 0x4 0xba 0xc 0x9b 0x0 0x0 0x0 0x1e 0x5 0x64 0x0 0x0 0x0 0x3 0x9 0x5d 0x0 0x13 0x1 0xd0 0x9 0x64 0x0 0x15 0x1 0xd7 0xff 0xdc 0x1 0xfa 0x13 0x80 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xfd 0x0 0x0 0x1 0xdb 0x1 0x52 0x1 0xb1 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x59 0xff 0xed 0x0 0x4e 0x3 0x8a 0x1 0xe 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x1 0x1e 0x0 0xe3 0xc 0xf7 0xc 0xeb 0x0 0x4 0x0 0xd1 0x79 0x96 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xd1 0xde 0x0 0x2e 0x0 0x0 0x5d 0x49 0x0 0x64 0x0 0x0 0x56 0xf2 0x0 0x31 0x0 0x0 0x4d 0x8d 0x0 0x1 0x0 0x0 0x44 0x16 0x0 0x5a 0x0 0x0 0xde 0x2e 0x0 0x48 0x0 0x0 0x28 0x49 0x0 0x58 0x0 0x0 0xaa 0x42 0x0 0x4a 0x27 0xf5 Cleanup recv buffer before send: 0xf7 0x4 0xba 0xc 0xb2 0x0 0x1 0x0 0x1f 0x5 0xb0 0x0 0x0 0x0 0x4 0x9 0x5d 0x0 0x13 0x1 0xce 0x9 0x5e 0x0 0x15 0x1 0xd9 0xff 0xdb 0x1 0xfc 0x13 0x7d 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xff 0x0 0x0 0x1 0xda 0x1 0x52 0x1 0xb1 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x5a 0xff 0xee 0x0 0x4e 0x3 0x8a 0x1 0xe 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x1 0x1e 0x0 0xe2 0xc 0xf7 0xc 0xeb 0x0 0x4 0x0 0xd1 0x79 0x98 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xd1 0xde 0x0 0x2e 0x0 0x0 0x5d 0x49 0x0 0x64 0x0 0x0 0x56 0xf2 0x0 0x31 0x0 0x0 0x4d 0x8d 0x0 0x1 0x0 0x0 0x44 0x16 0x0 0x5a 0x0 0x0 0xde 0x2e 0x0 0x48 0x0 0x0 0x28 0x49 0x0 0x58 0x0 0x0 0xaa 0x42 0x0 0x4a 0xfd 0xd2 Cleanup recv buffer before send: 0xf7 0x4 0xba 0xd 0x12 0x0 0x0 0x0 0x1b 0x5 0x43 0x0 0x0 0x0 0x5 0x9 0x5d 0x0 0x13 0x1 0xcf 0x9 0x60 0x0 0x14 0x1 0xd5 0xff 0xdb 0x1 0xf9 0x13 0x7f 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xfc 0x0 0x0 0x1 0xe3 0x1 0x52 0x1 0xb1 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x59 0xff 0xed 0x0 0x4e 0x3 0x8a 0x1 0xd 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x1 0x1f 0x0 0xe2 0xc 0xf7 0xc 0xea 0x0 0x4 0x0 0xd1 0x79 0x99 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xd1 0xde 0x0 0x2e 0x0 0x0 0x5d 0x49 0x0 0x64 0x0 0x0 0x56 0xf2 0x0 0x31 0x0 0x0 0x4d 0x8d 0x0 0x1 0x0 0x0 0x44 0x16 0x0 0x5a 0x0 0x0 0xde 0x2e 0x0 0x48 0x0 0x0 0x28 0x49 0x0 0x58 0x0 0x0 0xaa 0x42 0x0 0x4a 0xa1 0xda Cleanup recv buffer before send: 0xf7 0x4 0xba 0xc 0xf7 0x0 0x0 0x0 0x1f 0x5 0x7a 0x0 0x0 0x0 0x3 0x9 0x5d 0x0 0x13 0x1 0xcf 0x9 0x61 0x0 0x14 0x1 0xd5 0xff 0xe4 0x1 0xf8 0x13 0x81 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xff 0xf5 0x0 0x0 0x1 0xd5 0x1 0x52 0x1 0xb1 0x0 0x0 0x0 0x0 0x17 0x70 0xe8 0x90 0x0 0x0 0x0 0x0 0x2 0x2 0x5 0x5 0x9 0x59 0xff 0xee 0x0 0x4e 0x3 0x8a 0x1 0xe 0xa 0x44 0x7 0x8 0x1 0xf4 0x1 0xf4 0x1 0x1f 0x0 0xe3 0xc 0xf7 0xc 0xea 0x0 0x4 0x0 0xd1 0x79 0x9a 0x0 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x53 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xd1 0xde 0x0 0x2e 0x0 0x0 0x5d 0x49 0x0 0x64 0x0 0x0 0x56 0xf2 0x0 0x31 0x0 0x0 0x4d 0x8d 0x0 0x1 0x0 0x0 0x44 0x16 0x0 0x5a 0x0 0x0 0xde 0x2e 0x0 0x48 0x0 0x0 0x28 0x49 0x0 0x58 0x0 0x0 0xaa 0x42 0x0 0x4a 0xc8 0x2d

This error originated from a custom integration.

Logger: custom_components.foxess_modbus.modbus_controller Source: custom_components/foxess_modbus/modbus_controller.py:221 Integration: FoxESS - Modbus (documentation, issues) First occurred: 3:30:54 PM (1 occurrences) Last logged: 3:30:54 PM

/dev/ttyUSB0 247 - 5 failed poll attempts: now not connected. Last error: filedescriptor out of range in select()

This error originated from a custom integration.

Logger: custom_components.foxess_em.fox.fox_modbus Source: custom_components/foxess_em/fox/fox_modbus.py:89 Integration: FoxESS - Energy Management (documentation, issues) First occurred: 2:30:29 AM (4 occurrences) Last logged: 3:30:18 PM

Error writing holding register - retry (1/5): Modbus Error: [Input/Output] No Response received from the remote slave/Unable to decode response

D

On Sat, 9 Dec 2023 at 11:50, Antony Male @.***> wrote:

Thanks! It's likely a different problem to last time. I'm afraid I don't have a serial devie here, so I'm having to develop this workaround by proxy.

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1848387018, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2DKQAKUSMB4JVNQSF3YIRF7LAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBYGM4DOMBRHA . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 9 months ago

Thanks! Are you sure you're running 1.8.0b5? We shouldn't be using serialposix.py any more.

That said, my replacement for serialposix.py might still have an issue on that line - I'll look into it.

djdarlo commented 9 months ago

Yep. Just checked. 1.8.0b5.

On Mon, 11 Dec 2023 at 19:20, Antony Male @.***> wrote:

Thanks! Are you sure you're running 1.8.0b5? We shouldn't be using serialposix.py any more.

That said, my replacement for serialposix.py might still have an issue on that line - I'll look into it.

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1850730570, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2EKZK62YCFPA67N25LYI5MHLAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJQG4ZTANJXGA . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 9 months ago

Right! I've had a chance to properly look at this, and it turns out that pyserial's attempt to use poll rather than select just addresses the read operation, and not the write operation.

So I've updated my workaround to also provide a write implementation which uses poll (completely untested again, as I don't have a serial device). Can you give v1.8.0b6 a try please?

djdarlo commented 9 months ago

Of course

On Thu, 14 Dec 2023 at 16:02, Antony Male @.***> wrote:

Right! I've had a chance to properly look at this, and it turns out that pyserial's attempt to use poll rather than select just addresses the read operation, and not the write operation.

So I've updated my workaround to also provide a write implementation which uses poll. Can you give v1.8.0b6 a try please?

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1856118563, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2BKWXKCGYUWAFMSIYDYJMPKVAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJWGEYTQNJWGM . You are receiving this because you were mentioned.Message ID: @.***>

djdarlo commented 9 months ago

Done and disabled my nightly restart. Will report back.

D

On Thu, 14 Dec 2023 at 16:02, Antony Male @.***> wrote:

Right! I've had a chance to properly look at this, and it turns out that pyserial's attempt to use poll rather than select just addresses the read operation, and not the write operation.

So I've updated my workaround to also provide a write implementation which uses poll. Can you give v1.8.0b6 a try please?

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1856118563, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2BKWXKCGYUWAFMSIYDYJMPKVAVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJWGEYTQNJWGM . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 9 months ago

Thank you!

canton7 commented 8 months ago

Hi, any news?

djdarlo commented 8 months ago

All good so far :)

On Sun, 17 Dec 2023 at 12:50, Antony Male @.***> wrote:

Hi, any news?

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1859162991, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2DHHBZOJ2PRNMWOUGLYJ3S75AVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJZGE3DEOJZGE . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 8 months ago

Good to hear, thanks!

canton7 commented 8 months ago

Hi, is it still holding up a week in?

djdarlo commented 8 months ago

Yep. All good. Think you’ve cracked it :)

On Sat, 23 Dec 2023 at 13:07, Antony Male @.***> wrote:

Hi, is it still holding up a week in?

— Reply to this email directly, view it on GitHub https://github.com/nathanmarlor/foxess_modbus/issues/457#issuecomment-1868291325, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5PC2ETEOMJR6U3RI53O6TYK3JS7AVCNFSM6AAAAAA64FNBUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYGI4TCMZSGU . You are receiving this because you were mentioned.Message ID: @.***>

canton7 commented 8 months ago

Fab, thank you! I'll get that released properly at some point.