Camera live stream with Synology DSM sometimes fails

Quentame commented 4 years ago

The problem

Using an Eufy Indoor Cam 2K throw the Synology Surveillance Station on dev, when I open the live stream it sometimes stops with an error.

On my local network:

2020-09-10 21:25:11 ERROR (stream_worker) [libav.mp4] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 31010 >= 31010
2020-09-10 21:25:11 ERROR (stream_worker) [homeassistant.components.stream.worker] Stream connection failed: rtsp://syno:500df87de59a6bc75b9138aef67a8694@LOCAL_IP:554/Sms=1.unicast
Traceback (most recent call last):
  File "/Users/USER/dev/home-assistant/core/homeassistant/components/stream/worker.py", line 45, in stream_worker
    _stream_worker_internal(hass, stream, quit_event)
  File "/Users/USER/dev/home-assistant/core/homeassistant/components/stream/worker.py", line 243, in _stream_worker_internal
    mux_video_packet(packet)  # mutates packet timestamps
  File "/Users/pollet/dev/home-assistant/core/homeassistant/components/stream/worker.py", line 173, in mux_video_packet
    buffer.output.mux(packet)
  File "av/container/output.pyx", line 204, in av.container.output.OutputContainer.mux
  File "av/container/output.pyx", line 227, in av.container.output.OutputContainer.mux_one
  File "av/container/core.pyx", line 257, in av.container.core.Container.err_check
  File "av/error.pyx", line 336, in av.error.err_check
av.error.ValueError: [Errno 22] Invalid argument: '<none>'; last error log: [mp4] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 31010 >= 31010

When out of my local network, I got this all the time, no stream at all (maybe wrong IP/host):

2020-09-11 12:52:14 INFO (MainThread) [homeassistant.components.stream] Started stream: rtsp://syno:7fd0e8cf95f211d2ec1179f3a8eb2e14@LOCAL_IP:554/Sms=1.unicast
2020-09-11 12:52:19 ERROR (stream_worker) [libav.tcp] Connection to tcp://LOCAL_IP:554?timeout=5000000 failed: Operation timed out
2020-09-11 12:52:19 ERROR (stream_worker) [homeassistant.components.stream.worker] Stream connection failed: rtsp://syno:7fd0e8cf95f211d2ec1179f3a8eb2e14@LOCAL_IP:554/Sms=1.unicast
Traceback (most recent call last):
  File "/Users/USER/dev/home-assistant/core/homeassistant/components/stream/worker.py", line 45, in stream_worker
    _stream_worker_internal(hass, stream, quit_event)
  File "/Users/USER/dev/home-assistant/core/homeassistant/components/stream/worker.py", line 62, in _stream_worker_internal
    container = av.open(stream.source, options=stream.options)
  File "av/container/core.pyx", line 354, in av.container.core.open
  File "av/container/core.pyx", line 225, in av.container.core.Container.__cinit__
  File "av/container/core.pyx", line 257, in av.container.core.Container.err_check
  File "av/error.pyx", line 336, in av.error.err_check
av.error.TimeoutError: [Errno 60] Operation timed out: 'rtsp://syno:7fd0e8cf95f211d2ec1179f3a8eb2e14@LOCAL_IP:554/Sms=1.unicast'; last error log: [tcp] Connection to tcp://LOCAL_IP:554?timeout=5000000 failed: Operation timed out

Environment

Home Assistant Core release with the issue: 0.116.0.dev0 2020-09-11 commit 5117a1684177e95697400ada52ecad7bcbc9c8bd
Last working Home Assistant Core release (if known): none
Operating environment (OS/Container/Supervised/Core): MacOS via venv
Integration causing this issue: synology_dsm
Link to integration documentation on our website: https://www.home-assistant.io/integrations/synology_dsm

Traceback/Error logs

up

Additional information

Camera platform added on dev with #39838, ping @shenxn

probot-home-assistant[bot] commented 4 years ago

stream documentation stream source _{^{(message by IssueLinks)}}

Quentame commented 4 years ago

Just seen #39850 that is the same issue when I try to stream on my local network, duplicates on this part. So I think we can unassign you @hunterjm

But when I'm out of my network, I think it's a different issue we can focus on, with @shenxn

probot-home-assistant[bot] commented 4 years ago

Hey there @hunterjm, @uvjustin, mind taking a look at this issue as its been labeled with an integration (stream) you are listed as a codeowner for? Thanks! _{^{(message by CodeOwnersMention)}}

probot-home-assistant[bot] commented 4 years ago

Hey there @hacf-fr, mind taking a look at this issue as its been labeled with an integration (synology_dsm) you are listed as a codeowner for? Thanks! _{^{(message by CodeOwnersMention)}}

uvjustin commented 4 years ago

@hunterjm Any idea why it would mux the same packet twice? We have the monotonicity check in there...

uvjustin commented 4 years ago

@hunterjm I think it might be from a rounding error when the packet rebases the time base: https://github.com/PyAV-Org/PyAV/blob/1fa8dd6a95336321feca69b66aebefb4f08d7e85/av/container/output.pyx#L216 I'll make a PR that uses the time base of the inputs and see if that fixes things.

uvjustin commented 4 years ago

@Quentame can you see if this PR fixes the local network problem? (The other problem seems unrelated.)

hunterjm commented 4 years ago

I went ahead and merged it since it won't really do any harm and he can then just update to the next dev docker build. Feel free to re-open this issue if it doesn't solve your problem.

Quentame commented 4 years ago

It works fine.

Thanks for being so responsive 😍

ruimarinho commented 3 years ago

@uvjustin I'm still getting this issue on version 2021.2.1. Any chance you could revisit this? Thank you!

2021-02-07 20:18:45 ERROR (recorder_save_worker) [libav.mp4] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 346578 >= 346545
2021-02-07 20:18:45 ERROR (recorder_save_worker) [root] Uncaught thread exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/src/homeassistant/homeassistant/components/stream/recorder.py", line 64, in recorder_save_worker
    output.mux(packet)
  File "av/container/output.pyx", line 204, in av.container.output.OutputContainer.mux
  File "av/container/output.pyx", line 227, in av.container.output.OutputContainer.mux_one
  File "av/container/core.pyx", line 257, in av.container.core.Container.err_check
  File "av/error.pyx", line 336, in av.error.err_check
av.error.ValueError: [Errno 22] Invalid argument: '/tmp/camera_nas_1_hall_20210207-201832.mp4'; last error log: [mp4] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 346578 >= 346545
Exception ignored in: 'av.container.output.OutputContainer.__dealloc__'
Traceback (most recent call last):
  File "av/container/output.pyx", line 25, in av.container.output.close_output
2021-02-07 20:18:53 ERROR (MainThread) [root] Uncaught exception
Traceback (most recent call last):
  File "av/container/output.pyx", line 25, in av.container.output.close_output
TypeError: 'NoneType' object is not iterable
TypeError: 'NoneType' object is not iterable

Edit: It might be worth saying I'm streaming and recording at the same time. Not sure if that could cause issues with the monotonic clock?

uvjustin commented 3 years ago

@ruimarinho This issue looks similar but is not the same as the original one here. Please open another issue. The problem you have seems interesting and might be some kind of race condition with the threads. I forgot exactly what the original code looks like and @allenporter is in the middle of putting through quite a few changes so we can see if you still have the problem after those go through.

ruimarinho commented 3 years ago

Thank you for the input @uvjustin. Added as https://github.com/home-assistant/core/issues/46299.

home-assistant / core