search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.19k stars 30.57k forks source link

STREAM issue playing local MP4 files via FFMPEG or Local File Camera #34239

Closed GaryOkie closed 4 years ago

GaryOkie commented 4 years ago

The problem

Any valid/verified MP4 file, including those created by camera.record, should be playable via FFMPEG camera integration specifying the file as input. What happens is that when attempting to live view the MP4 stream, the stream playback will either hang (loops forever) and show nothing but an empty gray screen or it will display a small amount of video (2-4) seconds then hang with a gray screen.

Environment

The actual problem apparently lies within STREAM, it is just the cameras that can playback MP4 as input via Stream that exhibit this issue.

Problem-relevant configuration.yaml

stream:
camera:
  - platform: ffmpeg    # virtual camera to play MP4 recordings 
    input:  "/config/www/recordings/amcrest_doorbell.mp4"
    name: "Front Door Motion Activity"
##or
  - platform: local_file
    file_path: "/config/www/recordings/amcrest_doorbell.mp4"

camera entities are defined in any lovelace card that supports cameras and able to display a live view.

Traceback/Error logs

Logger: homeassistant.components.stream.worker
Source: components/stream/worker.py:82
Integration: stream
Error demuxing stream: no dts in packet

when this error occurs, the result is a hung gray screen in video playback

Additional information

I've tried using FFMPEG command line mode to post-process the recorded MP4 file using various encoding options, including disabling audio, but I haven't found out yet what STREAM is having a problem with.

GaryOkie commented 4 years ago

Hi @hunterjm - sorry to ping you directly, but I noticed in the last 2 cases where this same issue was reported, no one was assigned and the stale bot closed them. I'm hoping that won't happen here.

hunterjm commented 4 years ago

Stream only handles live feeds. A recorded file is not a live feed, so it will break as soon as it hits the end of the recording, which starts when HA starts, and probably ends before you can pull it up in lovelace. This type of functionality isn't supported.

GaryOkie commented 4 years ago

Live view/recorded view - is there really that big of a difference in the video stream except one has a finite end?

It would be great if an MP4 recording could be recognized by STREAM and just stop the stream at the end and pause with the last frame. Rewind/play again should work too. Actually, it's already doing that to some extent. Try it! I notice the longer the recording is, often, but not always, the more playback there is and it can be rewound and played again. Just the way it ends with the DTS error and gray screen is bad and I was hoping it could be fixable instead of just not supported.

Should I change this to an enhancement request then? Or should I change this to an FFMPEG camera issue? Issue: FFMPEG Live view should not send MP4 playback to STREAM for display, but process it differently because stream can't handle it? That sounds like a dead-end proposition.

Do you know of any other alternatives of how else can we play MP4 recordings in HA if not via STREAM?

hunterjm commented 4 years ago

That would be good. Here is where to implement it: https://github.com/home-assistant/core/blob/dev/homeassistant/components/stream/worker.py#L77

Right now if we get an EOS, we end the worker thread reading it. You could instead trigger it to restart. The "pause at last frame" won't really work due to how HLS works. We only save the last 3 "segments" which is why you only get limited playback today.

Either way, this is indeed an enhancement request, and not a bug.

You (or someone else) could also write a custom lovelace card to handle it, since MP4 is supported by default in many browsers implementation of the <video> element. I at one point was thinking of writing a custom "security" panel to handle this, but never found the time.

GaryOkie commented 4 years ago

Thanks Jason for the advice and direction. I hadn't expected you to point me to the actual source to fix it, uh, I mean enhance it, but I'll take a shot at it!

The <video> element in a custom card is also a very good idea. I've not created a card before, but I'll look into it as well.

hunterjm commented 4 years ago

I’d lean toward the Lovelace solution, unless you expect to actually do something with the “recording” entity in automations/scripts. If it’s purely for viewing the latest recording, it can be handled much more gracefully by just saving it to www and loading it in the frontend.

You could also just use a link in the frontend to have it open in a new tab, which Chrome will auto-play.

GaryOkie commented 4 years ago

ok, will do - and thanks again for the advice. I had come to the same conclusion after spending the past half hour studying the worker code! It was going to take a lot of logger inserts to figure it out.

hunterjm commented 4 years ago

@GaryOkie - Just did a quick test in a lovelace dashboard, and the Webpage or iframe card works great in both Chrome and the Home Assistant iOS app. So long as you place the .mp4 in www.

https://www.home-assistant.io/lovelace/iframe/

type: iframe
url: 'https://my.home.assistant/local/motion/foyer_latest.mp4'
aspect_ratio: 55%
title: Latest Foyer Motion
GaryOkie commented 4 years ago

Great! Thanks so much for point out this slick solution!

EDIT: Just added the "webpage" card to my security page, and it works nicely! The local MP4 plays to the end and stays there until the play button is tapped to replay. I'm not sure I need the snapshot of the last activity any more.

GaryOkie commented 4 years ago

Hi @hunterjm ,

Sorry, I now LoveLess this Lovelace mp4 playback solution because of the browser cache stuck on old content unless ctrl-5. It is important for security that the last activity from movement be the actual movement, not something that happened yesterday.

The best thing about creating an ffmpeg or local file camera using mp4 input was that stream has a great feature to refresh the snapshots in lovelace every 10 seconds from the stream.

I'm scouring the forums looking for ways to automatically force the browser to refresh. I found I can add a ?v=xxx to the URL tricking browser to pull in the new content, but how to automatically update the lovelace url with a unique xxx every 10 seconds or so is a trick I haven't learned how to do yet.

I know this isn't your issue, but just wanted to let you know the webpage card tip to play mp4 has a big downside that makes it practically unusable for me as a security notifier in lovelace (for now!).

GaryOkie commented 4 years ago

Hi @hunterjm -

Just a quick update to let you know that I solved the browser cache/refresh issue with playing mp4 via iframe card. I'm using the config-template-card to automatically modify the URL with a "?v=[timestamp entity].

This card utilizes webhooks to detect a state change in the timestamp entity which tracks the completion of the mp4 recording, and then immediately updates the URL in the iframe card. The lovelace view is then refreshed immediately as well. Perfect!

For anyone landing here looking at this issue - see this post.