search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.2k stars 30.57k forks source link

rtsp stream stops working in frontend with homeassistant.components.stream.worker warning in the main log #41891

Closed hioctane61 closed 4 years ago

hioctane61 commented 4 years ago

The problem

RTSP stream stops working after around 30 seconds in the frontend.

The stream works perfectly fine in VLC outside of homeassistant

Environment

arch | x86_64 chassis | desktop dev | false docker | true docker_version | 19.03.13 hassio | true host_os | Debian GNU/Linux 10 (buster) installation_type | Home Assistant Supervised os_name | Linux os_version | 5.7.0-0.bpo.2-amd64 python_version | 3.8.5 supervisor | 247 version | 0.116.3 virtualenv | false

The device is an Amcrest AD110 doorbell camera I know that this was working fine in 0.115.x, no errors, working very well. Something changed in 0.116.x and this issue manifested itself.

stream:
amcrest:
  - host: 192.168.86.209
    name: Doorbell Camera
    username: !secret amcrest_username
    password: !secret amcrest_pw
    stream_source: rtsp
    binary_sensors:
      - motion_detected
      - online

Traceback/Error logs

2020-10-15 15:56:11 WARNING (stream_worker) [homeassistant.components.stream.worker] Timestamp overflow detected: dts = 632496, resetting stream
2020-10-15 15:56:13 WARNING (stream_worker) [homeassistant.components.stream.worker] Dropping out of order packet: 24300 <= 24300

Additional information

uvjustin commented 4 years ago

The stream is resetting because we are detecting a timestamp overflow. I'm not sure why this is happening - this may be a spurious error or related to some kind of network issue or stream reset. This PR might have addressed it by allowing for 2 of these packets. https://github.com/home-assistant/core/pull/41467 . Please ignore the other out of order packet message. It's a warning, not an error, and it was removed in the PR I linked.

hioctane61 commented 4 years ago

Thanks for swift reply. I keep watch on that pull request, if it makes it into near future release, hopefully it will resolve issue. I can then close.

uvjustin commented 4 years ago

@hunterjm Can we cherry pick https://github.com/home-assistant/core/pull/41467 ?

probot-home-assistant[bot] commented 4 years ago

Hey there @hunterjm, mind taking a look at this issue as its been labeled with an integration (stream) you are listed as a codeowner for? Thanks! (message by CodeOwnersMention)

tachikoma1373 commented 4 years ago

seeing the same issue:

Logger: homeassistant.components.stream.worker Source: components/stream/worker.py:240 Integration: Stream (documentation, issues) First occurred: 11:05:29 (2 occurrences) Last logged: 12:35:15

Timestamp overflow detected: dts = 245068, resetting stream Timestamp overflow detected: dts = 314555, resetting stream

it seems to be causing camera.record to record shortened videos for my security cams

uvjustin commented 4 years ago

Actually I think I found the issue, making a PR to fix this now.

hioctane61 commented 4 years ago

Actually I think I found the issue, making a PR to fix this now.

That's awesome. Thank you.

uvjustin commented 4 years ago

@hioctane61 I'm not sure it will fix your problem. If not, there are a few other things we can try. The hard part with troubleshooting a component like this is that everyone's setup is different so I can't reproduce the problem to work on a fix. Are you familiar enough with HA to try custom code/custom components? That can shorten the feedback cycle as we can try a few different things without waiting for a new HA release each time.

hioctane61 commented 4 years ago

@uvjustin I don't mind taking the 'peel the onion' approach. however, if you can advise on how the stream integration can be customised, that be great. i could try some different things. One thing though, it definitely started with 0.116. So if we can thread the needle of keeping those changes and avoiding stream resets, that would do it

DGAzr commented 4 years ago

I'm able to reproduce this pretty reliably with my Amcrest WiFi camera configured via the ONVIF integration. No more than 5 minutes of the stream. It definitely seems to have only started with 0.116 as I've been using this particular camera to monitor my toddlers for a year or so.

I am happy to try out custom components to confirm a fix if that is helpful.

Errors from logs: 2020-10-18 12:26:21 WARNING (stream_worker) [homeassistant.components.stream.worker] Timestamp overflow detected: dts = 17504280, resetting stream 2020-10-18 12:26:22 ERROR (MainThread) [homeassistant] Error doing job: Unclosed client session 2020-10-18 12:26:22 ERROR (MainThread) [homeassistant] Error doing job: Unclosed client session 2020-10-18 12:26:41 WARNING (stream_worker) [homeassistant.components.stream.worker] Timestamp overflow detected: dts = 1769670, resetting stream 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65544 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65546 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65548 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65550 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65552 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65554 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65556 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65558 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65560 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65562 2020-10-18 12:27:05 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65564 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65566 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65568 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65570 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65572 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65574 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65576 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65578 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65580 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65582 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65584 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65586 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65588 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65590 2020-10-18 12:27:06 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65592 2020-10-18 12:27:08 WARNING (stream_worker) [homeassistant.components.stream.worker] Dropping out of order packet: 193085 <= 193085 2020-10-18 12:27:23 ERROR (SyncWorker_38) [homeassistant.components.onvif] Fetch snapshot image failed from Living Room Camera, falling back to FFmpeg; HTTPConnectionPool(host='192.168.99.36', port=80): Read timed out. (read timeout=5) 2020-10-18 12:28:45 WARNING (MainThread) [haffmpeg.tools] Timeout reading image. 2020-10-18 12:28:53 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65542 2020-10-18 12:28:53 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65544 2020-10-18 12:28:53 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65546 2020-10-18 12:28:53 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65548 2020-10-18 12:28:53 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65550 2020-10-18 12:28:53 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65552 2020-10-18 12:28:53 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65554 2020-10-18 12:28:53 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65556 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65558 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65560 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65562 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65564 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65566 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65568 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65570 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65572 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65574 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65576 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65578 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65580 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65582 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65584 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65586 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65588 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65590 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65592 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65594 2020-10-18 12:28:54 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65596 2020-10-18 12:28:55 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65598 2020-10-18 12:28:55 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65600 2020-10-18 12:28:55 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65602 2020-10-18 12:28:55 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65604 2020-10-18 12:28:55 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65606 2020-10-18 12:28:55 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65608 2020-10-18 12:28:55 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65610 2020-10-18 12:28:55 ERROR (stream_worker) [libav.h264] Missing reference picture, default is 65612 2020-10-18 12:28:56 WARNING (stream_worker) [homeassistant.components.stream.worker] Dropping out of order packet: 174351 <= 174351

uvjustin commented 4 years ago

I'm able to reproduce this pretty reliably with my Amcrest WiFi camera configured via the ONVIF integration. No more than 5 minutes of the stream. It definitely seems to have only started with 0.116 as I've been using this particular camera to monitor my toddlers for a year or so.

I am happy to try out custom components to confirm a fix if that is helpful.

Great. Can you try https://github.com/home-assistant/core/pull/41951 and see if it helps?

erichiller commented 4 years ago

Same here, I don't have a boatload of free time to text custom components, but if there is a strong need I will be happy to help.

Just to add to the info:

Logger: homeassistant.components.stream.worker
Source: components/stream/worker.py:240
Integration: Stream (documentation, issues)
First occurred: 8:02:09 AM (1 occurrences)
Last logged: 8:02:09 AM

Timestamp overflow detected: dts = 230960, resetting stream

It's an Amcrest 1051. running on Ethernet/wired.

hioctane61 commented 4 years ago

@uvjustin dude you fixed it! thanks a million.