[Feature Request] HomeKit Video Support

[moralmunky]

Is your feature request related to a problem? Please describe. USPS Mail camera shows in HomeKit but can't rotate images because the file is not an MP4

Describe the solution you'd like Generate a GIF and/or MP4 to display the mail images.

[firstof9]

Interesting, shouldn't be too hard.

[firstof9]

We can use ffmpeg to do this: ffmpeg -f gif -i infile.gif outfile.mp4

[firstof9]

Likely have a toggle bool in the config flow to turn this feature on/off I'd assume as well.

[firstof9]

Did some untested work on this last night if you wanna give it a shot.

Untested gif-to-mp4

[firstof9]

Tested it, my latest commits work, tho the images seem to be looking scrambled.

[moralmunky]

You just take it and run with it! I do something similar with weather radar maps. I’ll see if the added ffmpeg options I use help.

[firstof9]

I'm wondering of for the images we need to use the message_from_bytes instead of message_from_string ?

[moralmunky]

What version of ffmpeg do you have on your system? I am getting errors running the ffmpeg command manually on this gif. Seems to be a bug in the ffmpeg code.

[gif @ 0x55b1a41f94e0] Invalid image height.
Error while decoding stream #0:0: Invalid data found when processing input
[gif @ 0x55b1a41f94e0] Invalid image width.
Error while decoding stream #0:0: Invalid data found when processing input

[moralmunky]

Installing v4.2.x and using the configuration I used for my weather maps produced a readable image. Though, there was one image out of four it did like and was all screwed up.

ffmpeg -f gif -i mail_today.gif -pix_fmt yuv420p -c:v libx264 -preset veryslow -tune animation -profile:v baseline -crf 34 -movflags +faststart -filter:v crop='floor(in_w/2)*2:floor(in_h/2)*2' mail_test.mp4

[firstof9]

I used whatever is installed by default in the Home Assistant docker image.

If we need to we could add a script that could be executed to compile/install the binary in the component directory for use with the component.

[firstof9]

Since it would be in the component directory it'd survive docker image updates.

[moralmunky]

So the main cause of the error is that the height of the image is 319 which is not divisible by 2. By making the gif 724x320 the older version ffmpeg (3.4.6-0 on my ubuntu0.18.04.1) works perfectly.

[firstof9]

That'd be an easy fix.

[firstof9]

I spun this up (resize images branch) and will be testing it shortly. I verified the requirement installs in the docker container without error.

[firstof9]

Tests completed, images get resized properly to 724x320

[firstof9]

And now I've merged the resize into the gif-to-mp4 branch.

[firstof9]

Found out if you only have 1 piece of mail the mp4 doesn't work right.

[moralmunky]

Tested this with additional ffmpeg options on a day with a single mail piece. Everything shows up for my set up.

    subprocess.call(
        ["ffmpeg", "-f", "gif", "-i", gif_image, "-pix_fmt", "yuv420p", "-filter:v", "crop='floor(in_w/2)*2:floor(in_h/2)*2'", mp4_file],
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
    )

[firstof9]

Nice I'll give it a test go in the morning.

[firstof9]

Just tested with the latest code, ran smooth as silk now. No glitching images either.

[moralmunky]

This all works great, Still need to figure out how to feed the video to HomeKit. It shows up but I either have issues with my installation and stream or I need to figure out how to loop it.

[moralmunky]

Trying some stuff to better support HomeKit.

Using the ffmpeg camera to send to HomeKit

- platform: ffmpeg
    input: '/home/homeassistant/.homeassistant/www/mail_and_packages/mail_today.mp4'
    name: mail_usps_mp4

Modified the b21 with these lines to extended the length of the last frame so it's not a flash then take that Mp4 and loop it x amount of times

Looping the first results in the last image of the gif just being a split second flash.

Take a look, see what you think.

https://github.com/moralmunky/Home-Assistant-Mail-And-Packages/blob/1bce849dca20e06345c614ecdb2799f173bb21cd/custom_components/mail_and_packages/sensor.py#L446-L511

[firstof9]

Should be able to condense that down. I'll check to see if there's any additional ways to help with this too.

[firstof9]

I haven't tested this yet but instead of 2 calls we might be able to just use:

    subprocess.call(
        [
            "ffmpeg",
            "-f",
            "gif",
            "-i",
            gif_image,
            "-pix_fmt",
            "yuv420p",
            "-filter:v",
            "crop='floor(in_w/2)*2:floor(in_h/2)*2'",
            "-filter_complex",
            "loop=2",
            mp4_file,
        ],
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
    )

Maybe 3 or 4 loops for HomeKit would be better i dunno

[parthmodi]

Hi I was wondering if this feature has more progress? I was able to get the mail to show on the camera. But whenever I tap the camera, it just loads forever. Also, not sure if there is a way to trigger the camera on homekit as if it were a doorbell or something?

[firstof9]

You setup the file camera in Home Assistant?

[parthmodi]

Yeah, I setup in my configuration.yaml. The camera works in home assistant I can view it. And in HomeKit, it shows up. But whenever I tap on the camera it goes into a buffering state.

[firstof9]

Ah, I don't think you can stream file cameras. You might be able to do a ffmpeg camera with the mp4, but I'm not 100% sure it's streamable.

[parthmodi]

I have the FFMPEG camera already. That is what I think is giving me the preview of the video. But not able to load. If there is any other way to make it work better, would be willing to try it out.

[firstof9]

Try this sensor.py see if it allows streaming.

[parthmodi]

No, this one made it so that mail does not update at all.

[firstof9]

It's based on the 0.2.2 version, so if you're running 0.2.1 it won't work correctly.

[madrobby]

I've turned the "Create MP4 from images" option on and don't see any MP4's created anywhere. Where are these supposed to be stored? I have the "ffmpeg:" line in my main configuration and confirmed that ffmpeg works on the terminal in my docker container. Is there anything else I need to configure? I can see the animated GIF in the Home Assistant dashboard, but HomeKit only shows me a "camera doesn't work" icon.

[firstof9]

The mp4 file should end up in the same directory as the gif. Please check your log for errors.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.