Runtime Error

[Coolie1101]

Home Assistant release (hass --version): 0.64.2

Python release (python3 --version): 3.5.3

Component/platform: Unsure, maybe Telegram?

Description of problem: Repeat error in log.

Expected: No Error's

Traceback (if applicable):

Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 38, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 55 is used by transport <_SelectorSocketTransport fd=55 read=polling write=<polling, bufsize=0>>

2018-03-04 10:47:39 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback SendfilePayloadWriter._sendfile_cb(<Future cancelled>, 46, 22, 27264, 7470, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 38, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 46 is used by transport <_SelectorSocketTransport fd=46 read=polling write=<polling, bufsize=0>>

EDIT: Seems to have been resolved since 0.65

[Coolie1101]

Still experiencing the same error repeatedly, which is blowing up the log.

Log size is 1,793 KB in just under 3 hours, can anyone guide me as to how I may be able to troubleshoot or correct this error?

First error below:

2018-03-20 11:40:54 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 48, 47, 83984, 2064396, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 36, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 48 is used by transport <_SelectorSocketTransport fd=48 read=polling write=<polling, bufsize=0>>

Then repeated almost every second, below are some error's with different descriptor numbers:

2018-03-20 12:16:14 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 51, 50, 70952, 2259024, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 36, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 51 is used by transport <_SelectorSocketTransport fd=51 read=polling write=<polling, bufsize=0>>

2018-03-20 14:12:18 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 53, 52, 69504, 603940, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 36, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 53 is used by transport <_SelectorSocketTransport fd=53 read=polling write=<polling, bufsize=0>>

[manuel-jrs]

I have the same error

Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 27, 21, 15004, 56266, <_UnixSelecto...e debug=False>, True)

Traceback (most recent call last):
  File "/usr/lib/python3.6/asyncio/events.py", line 127, in _run
    self._callback(*self._args)
  File "/home/orangepi/homeassistant/lib/python3.6/site-packages/aiohttp/web_fileresponse.py", line 36, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.6/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.6/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 27 is used by transport <_SelectorSocketTransport fd=27 read=polling write=<polling, bufsize=0>>

I'm not sure if it is related to #13129

[oldfart101]

me too

ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 18, 16, 27740, 161715, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 36, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 18 is used by transport <_SelectorSocketTransport fd=18 read=polling write=<polling, bufsize=0>>
2018-04-01 18:06:27 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 18, 16, 27740, 161715, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 36, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))

[Coolie1101]

@manuel-jrs @oldfart101 Which notify component are you guy's using?

[oldfart101]

don't understand the question

[Coolie1101]

Are you sending messages/images from HA to somewhere?

[manuel-jrs]

@Coolie1101 I'm using telegram platform.

[Coolie1101]

@manuel-jrs Same here, did you upgrade to 0.66.0 and still seeing the same errors?

[oldfart101]

only MQTT, errors still there after upgrade to 0.66.0

[Coolie1101]

@oldfart101 Thanks, thought it was only related to Telegram.

Did you upgrade to 0.66.0 and still seeing the same errors?

[oldfart101]

yes, P.S. does Alexa via cloud count?

[manuel-jrs]

@Coolie1101 I upgraded it to 0.66 and still the same errors.

[lucaspnw]

Same problem here, pretty barebones install. Just a single zwave USB stick and a few zwave PIR sensors set up. no automation, no triggers/events. Running hassbian on a RPi-2b.

Version: 0.66.1

It started after about 24 hours total uptime.

Apr 10 20:08:30 hassbian hass[426]: INFO:homeassistant.core:Bus:Handling , entity_id=sun.sun, new_state=>
Apr 10 20:09:30 hassbian hass[426]: INFO:homeassistant.core:Bus:Handling , entity_id=sun.sun, new_state=>
Apr 10 20:10:19 hassbian hass[426]: ERROR:homeassistant.core:Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(, 16, 15, 59860, 12367, <_UnixSelecto...e debug=False>, True)
Apr 10 20:10:19 hassbian hass[426]: Traceback (most recent call last):
Apr 10 20:10:19 hassbian hass[426]:   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
Apr 10 20:10:19 hassbian hass[426]:     self._callback(*self._args)
Apr 10 20:10:19 hassbian hass[426]:   File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 36, in _sendfile_cb
Apr 10 20:10:19 hassbian hass[426]:     loop.remove_writer(out_fd)
Apr 10 20:10:19 hassbian hass[426]:   File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
Apr 10 20:10:19 hassbian hass[426]:     self._ensure_fd_no_transport(fd)
Apr 10 20:10:19 hassbian hass[426]:   File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
Apr 10 20:10:19 hassbian hass[426]:     fd, transport))
Apr 10 20:10:19 hassbian hass[426]: RuntimeError: File descriptor 16 is used by transport <_SelectorSocketTransport fd=16 read=polling write=>
Apr 10 20:10:19 hassbian hass[426]: INFO:homeassistant.components.http.view:Serving /api/websocket to 192.168.0.69 (auth: True)
Apr 10 20:10:19 hassbian hass[426]: INFO:homeassistant.core:Bus:Handling , 16, 15, 59860, 12367, <_UnixSelecto...e debug=False>, True), timestamp=1523391019.5628154, source=/srv/homeassistant/lib/python3.5/site-packages/homeassistant/core.py, exception=Traceback (most recent call last):
Apr 10 20:10:19 hassbian hass[426]:   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
Apr 10 20:10:19 hassbian hass[426]:     self._callback(*self._args)
Apr 10 20:10:19 hassbian hass[426]:   File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 36, in _sendfile_cb
Apr 10 20:10:19 hassbian hass[426]:     loop.remove_writer(out_fd)
Apr 10 20:10:19 hassbian hass[426]:   File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
Apr 10 20:10:19 hassbian hass[426]:     self._ensure_fd_no_transport(fd)
Apr 10 20:10:19 hassbian hass[426]:   File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
Apr 10 20:10:19 hassbian hass[426]:     fd, transport))
Apr 10 20:10:19 hassbian hass[426]: RuntimeError: File descriptor 16 is used by transport <_SelectorSocketTransport fd=16 read=polling write=>
Apr 10 20:10:19 hassbian hass[426]: >
Apr 10 20:10:19 hassbian hass[426]: ERROR:homeassistant.core:Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(, 16, 15, 59860, 12367, <_UnixSelecto...e debug=False>, True)

[Coolie1101]

Still going. Could it be because aiohttp==3.0.6?

Anyone know what "RuntimeError: File descriptor ##" means or refers to?

Tue Apr 10 2018 22:37:28 GMT-0400 (Eastern Daylight Time)

Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 52, 51, 70952, 1909800, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 36, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 52 is used by transport <_SelectorSocketTransport fd=52 read=polling write=<polling, bufsize=0>>

[MaximumSU]

Same problem. No Telegram. Version: 0.67.1

[oldfart101]

Upgrade to 0.67.1 log file is 1680KB after an hour

2018-04-25 21:41:38 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 17, 15, 27740, 162341, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 37, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 17 is used by transport <_SelectorSocketTransport fd=17 read=polling write=<polling, bufsize=0>>
2018-04-25 21:41:39 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 17, 15, 27740, 162341, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 37, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))

[Joezi79]

Same problem after a fresh installation. Version is 0.68.1. No Telegram running. Anyone an idea.

[nlienard]

same issue after updating to 0.68.1

After stop, disabling influxdb, start, no more error. then i re-enabled influxdb, and also no error.

[jonny190]

Did anyone solve this i'm getting the same

[MaximumSU]

Ver. 0.70.0 Not fixed yet. The error is spontaneous. Only one problem because of it - my log file blowing up to 17Mb at day....

[rotcop4u2]

I am getting the same error. i went 48 hours with out it last time, but then it hit hard. the only way i could stop it was to reboot my RPI. Log got almost 700MB in 15 minutes.

Jun 11 05:22:47 HomeAssistant hass[701]: RuntimeError: File descriptor 19 is used by transport <_SelectorSocketTransport fd=19 read=polling write=<polling, bufsize=0>>
Jun 11 05:22:47 HomeAssistant hass[701]: ERROR:homeassistant.core:Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 19, 18, 27740, 54642, <_UnixSelecto...e debug=False>, True)
Jun 11 05:22:47 HomeAssistant hass[701]: Traceback (most recent call last):
Jun 11 05:22:47 HomeAssistant hass[701]:   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
Jun 11 05:22:47 HomeAssistant hass[701]:     self._callback(*self._args)
Jun 11 05:22:47 HomeAssistant hass[701]:   File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 37, in _sendfile_cb
Jun 11 05:22:47 HomeAssistant hass[701]:     loop.remove_writer(out_fd)
Jun 11 05:22:47 HomeAssistant hass[701]:   File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
Jun 11 05:22:47 HomeAssistant hass[701]:     self._ensure_fd_no_transport(fd)
Jun 11 05:22:47 HomeAssistant hass[701]:   File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
Jun 11 05:22:47 HomeAssistant hass[701]:     fd, transport))

[bbrendon]

I just noticed this for the first time today. It doesn't appear to have happened prior to 0.73.2. Other versions I was on recently:

0.73.1 - I was on briefly. 0.71.0 - for quite a long time.

[Ithanil]

I see the same error filling a nearly 10MB log file in 0.71.0

[dodepier]

Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 41, 40, 26752, 55686, <_UnixSelecto...e debug=False>, True)
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/srv/homeassistant/lib/python3.5/site-packages/aiohttp/web_fileresponse.py", line 37, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 41 is used by transport <_SelectorSocketTransport fd=41 read=polling write=<polling, bufsize=0>>

I have the same error

Home Assistant - 0.78.0

[aosadchyy]

Same issue happens in HA 0.79. Apparently I get this outburst of errors when logging into HA Web UI. HA tries to load additional components, as per this thread: https://github.com/home-assistant/architecture/issues/52 From my HA logs:

2018-09-30 13:51:40 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback SendfileStreamWriter._sendfile_cb(<Future cancelled>, 20, 18$
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
    self._callback(*self._args)
  File "/usr/local/lib/python3.5/dist-packages/aiohttp/web_fileresponse.py", line 37, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 351, in remove_writer
    self._ensure_fd_no_transport(fd)
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 258, in _ensure_fd_no_transport
    fd, transport))
RuntimeError: File descriptor 20 is used by transport <_SelectorSocketTransport fd=20 read=polling write=<polling, bufsize=0>>
2018-09-30 13:51:43 INFO (MainThread) [homeassistant.loader] Loaded hangouts from homeassistant.components.hangouts
2018-09-30 13:51:43 INFO (MainThread) [homeassistant.loader] Loaded nest from homeassistant.components.nest
2018-09-30 13:51:43 INFO (MainThread) [homeassistant.loader] Loaded openuv from homeassistant.components.openuv
2018-09-30 13:51:44 INFO (MainThread) [homeassistant.loader] Loaded cast from homeassistant.components.cast
2018-09-30 13:51:44 INFO (MainThread) [homeassistant.loader] Loaded deconz from homeassistant.components.deconz
2018-09-30 13:51:44 INFO (MainThread) [homeassistant.loader] Loaded ios from homeassistant.components.ios

[mykola-dev]

Hass.io 0.79.3 Not using telegram It hangs whole cpu core on my raspberry (100% load)

Traceback (most recent call last):
  File "uvloop/cbhandles.pyx", line 66, in uvloop.loop.Handle._run
  File "/usr/local/lib/python3.6/site-packages/aiohttp/web_fileresponse.py", line 37, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "uvloop/loop.pyx", line 2272, in uvloop.loop.Loop.remove_writer
  File "uvloop/loop.pyx", line 787, in uvloop.loop.Loop._remove_writer
  File "uvloop/loop.pyx", line 692, in uvloop.loop.Loop._ensure_fd_no_transport
RuntimeError: File descriptor 23 is used by transport <ReadUnixTransport closed=False reading=True 0x72ce0eb0>
2018-10-07 23:18:15 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback <bound method SendfileStreamWriter._sendfile_cb of <aiohttp.web_fileresponse.SendfileStreamWriter object at 0x7056deb0>>

I think this is happening when i restart my router. I will try to reproduce it later.

[andreipopovici]

I get the same issue on 0.79.1, followed by a full HA crash, along with:

python: src/unix/core.c:896: uv__io_stop: Assertion `loop->watchers[w->fd] == w' failed.

Googling that particular aiohttp error didn't turn up a fix.

Edit: I do have the camera component loaded and displaying an image on the Lovelace dashboard.

[mykola-dev]

This is ~ridiculous~ sad. Today morning i got spammed again. My 32gb card is 70% full and CPU is overheated (72*). The error log message appears in the log several hundreds times per second!

Btw, i have cameras in my setup and Lovelace enabled too. I can provide the configs.

[ualex73]

Same problem here, it filled up 40GB of diskspace within 8 hours.

Millions of the following line:

2018-10-10 14:56:04 ERROR (MainThread) [homeassistant.core] Error doing job: Exception in callback <bound method SendfileStreamWriter._sendfile_cb of <aiohttp.web_fileresponse.SendfileStreamWriter
object at 0x7f9c51d9ee80>>
Traceback (most recent call last):
  File "uvloop/cbhandles.pyx", line 66, in uvloop.loop.Handle._run
  File "/usr/local/lib/python3.6/site-packages/aiohttp/web_fileresponse.py", line 37, in _sendfile_cb
    loop.remove_writer(out_fd)
  File "uvloop/loop.pyx", line 2272, in uvloop.loop.Loop.remove_writer
  File "uvloop/loop.pyx", line 787, in uvloop.loop.Loop._remove_writer
  File "uvloop/loop.pyx", line 692, in uvloop.loop.Loop._ensure_fd_no_transport
RuntimeError: File descriptor 36 is used by transport <ReadUnixTransport closed=False reading=True 0x7f9c51da9d48>

[awarecan]

Caused by aio-libs/aiohttp#1893

[awarecan]

Need someone help to test following patch, the location of your web_fileresponse.py can be found in the error log

--- aiohttp/web_fileresponse.py
+++ aiohttp/web_fileresponse.py
@@ -33,11 +33,12 @@ def _write(self, chunk):

     def _sendfile_cb(self, fut, out_fd, in_fd,
                      offset, count, loop, registered):
-        if registered:
-            loop.remove_writer(out_fd)
         if fut.cancelled():
             return

+        if registered:
+            loop.remove_writer(out_fd)
+
         try:
             n = os.sendfile(out_fd, in_fd, offset, count)
             if n == 0:  # EOF reached

[ualex73]

I will apply it to my Home Assistant, only i had this issue only 3 times - thus it isn't something i can easily reproduce.

[mykola-dev]

Need someone help to test following patch, the location of your web_fileresponse.py can be found in the error log

Should i replace it inside of docker container if i use Hass.io? /usr/local/lib/python3.6/site-packages/aiohttp/web_fileresponse.py

It is reproduceable on my environment pretty easy when i play with my cameras setup. I believe one of the cameras has awful hardware and produces timeouts very often, so http client is going crazy after that. I'll try the patch on near weekend

[awarecan]

@deviant-studio what camera are you using?

[mykola-dev]

@awarecan i have several, but suspect this one https://ru.aliexpress.com/item/BESDER-Yoosee-IP-Camera-Wifi-1080P-960P-720P-ONVIF-Wireless-Wired-P2P-CCTV-Bullet-Outdoor-Camera/32793309300.html since its resolving time is the worst (until image appears in the card), and sometimes image doesn't appear at all. in these moments my hass instance is going crazy and spams the log with errors. I use all my cameras in jpeg mode, like http://url-to-jpeg-image

[awarecan]

I mean what HA camera platform are you using? https://www.home-assistant.io/components/camera.generic/?

[mykola-dev]

@awarecan yes, all my cameras are generics

upd: sorry for misinformation. the problematic camera is running under ffmpeg platform. i was unable to run it under generic or onvif (despite it supports onvif) here is the config

- platform: ffmpeg
  input: 'rtsp://192.168.1.111:554/onvif2'
  name: outdoor 3 

[dodepier]

Need someone help to test following patch, the location of your web_fileresponse.py can be found in the error log

--- aiohttp/web_fileresponse.py
+++ aiohttp/web_fileresponse.py
@@ -33,11 +33,12 @@ def _write(self, chunk):

     def _sendfile_cb(self, fut, out_fd, in_fd,
                      offset, count, loop, registered):
-        if registered:
-            loop.remove_writer(out_fd)
         if fut.cancelled():
             return

+        if registered:
+            loop.remove_writer(out_fd)
+
         try:
             n = os.sendfile(out_fd, in_fd, offset, count)
             if n == 0:  # EOF reached
``

It did change the code no error so far. However, I do not know what did trigger the error. So I have to wait for a couple of days

[signaleleven]

I just noticed the same. No idea what triggered it.

[awarecan]

@signaleleven could you please try to apply my patch?

[bbrendon]

Does this issue only occur for people with cameras in the HASS GUI? I have a few cameras, but I almost never click the Tab in the GUI where they are located which might be why this issue has only appeared for me once (that I have noticed anyway).

[signaleleven]

In my case, yes, I have some cameras in the gui, and one is not working (for other reasons). I will try to reproduce it. If I manage to reproduce it reliably, I will apply the patch.

[signaleleven]

@awarecan It happened again (good thing I set up an alarm on protracted high CPU usage :) ) and I applied your patch. I use hassio and I had to edit the file within the container. A simple container restart preserved the change. It will not survive a container recreation but until the next upgrade it should hold.

[pagenoare]

@awarecan just wanted to confirm that your patch has worked perfectly for me. Thanks much!

[xatr0z]

I dont have any camera's and still have the same error. Didnt saw it before, cant think of anything that triggered it (only tried some light automations today).

I will try the patch

[mstovenour]

Need someone help to test following patch, the location of your web_fileresponse.py can be found in the error log

--- aiohttp/web_fileresponse.py
+++ aiohttp/web_fileresponse.py
@@ -33,11 +33,12 @@ def _write(self, chunk):

     def _sendfile_cb(self, fut, out_fd, in_fd,
                      offset, count, loop, registered):
-        if registered:
-            loop.remove_writer(out_fd)
         if fut.cancelled():
             return

+        if registered:
+            loop.remove_writer(out_fd)
+
         try:
             n = os.sendfile(out_fd, in_fd, offset, count)
             if n == 0:  # EOF reached

This worked for my new installation. I had the same errors, probably related to how I have pages open on my ipad and iphone. None since applying this change.

[rotcop4u2]

@mstovenour what do I remove or add? this is what I have

 def _sendfile_cb(self, fut, out_fd, in_fd,
                     offset, count, loop, registered):
        if registered:
            loop.remove_writer(out_fd)
        if fut.cancelled():
            return

        try:
            n = os.sendfile(out_fd, in_fd, offset, count)
            if n == 0:  # EOF reached
                n = count
        except (BlockingIOError, InterruptedError):
            n = 0
        except Exception as exc:
            fut.set_exception(exc)
            return

        if n < count:
            loop.add_writer(out_fd, self._sendfile_cb, fut, out_fd, in_fd,
                            offset + n, count - n, loop, True)
        else:
            fut.set_result(None)

        else:
            fut.set_result(None)

[mykola-dev]

On 81.1 i don't have errors in the log so far. but still got spontaneous cpu full loads, and the only cure is to restart. Can't say for sure if it's same issue since my log is empty.

[signaleleven]

I also don't have the flood of logging anymore (I think already in 0.8), but I noticed that when the CPU jumps to 100% it INVARIABLY goes back to normal after exactly 2 hours. And at the same time I get exactly ONE message in the log.

I applied the patch but of course I lost it at every upgrade, so I am living with the problem for now (this made me jump from a Rpi to an Intel NUC so I can survive)

(I just noticed that in my screenshot above the CPU usage goes down after more than 2 hours. But I must have retriggered the problem by browsing the UI while the problem was present. If I leave the UI alone, it's always going back to nominal in about 2 hours)

[signaleleven]

@mstovenour what do I remove or add? this is what I have

You have to invert the two blocks

     if registered:
         loop.remove_writer(out_fd)

and

        if fut.cancelled():
            return