Homekit Secure Video

[TechRemarker]

In June, Apple announced the upcoming Homekit Secure Video feature where all video will also be backed up in iCloud. While I assume it will have to be finalized before we know for sure, but in theory will this plugin be able to support that, or do we think that feature will only work with manufacturer's working directly with Apple?

[mneumark]

Agreed. The keynote only gave more reason to want this feature. Not less.

[donavanbecker]

sorry closed on accident.

[vkedwardli]

https://twitter.com/HerrSupereg/status/1214025078603755520 https://github.com/Supereg/secure-video-specification

There is an unofficial spec (in case you don't know)

[llemtt]

@vkedwardli those specifications confirm that it's actually beyond ffmpeg capabilities (HDS protocol)

another video processor must be used, maybe a custom version of ffmpeg or a pipe into another process implementing HDS (this could be done also inside the plugin eventually)

[hazcod]

I mentioned it in https://github.com/brutella/hc/issues/174 Imagine the possibilities :-)

[monza258]

Any news about secure video?

[california444]

@Supereg

At first thanks for the HomeKit secure video specification! It looks like that it is possible to create fragmented mp4 files with ffmpeg. I created the files here with ffmpeg from my camera. I looks like that they confirm to your documentation (init and media fragments, 4 seconds long, init file contains ftyp and moov, segments contain moof and mdata according to http://mp4parser.com, each segment starts with a keyframe....) and they are playable in Safari browser.

Are you somehow capable to confirm that the format of the created files is correct and that it is worth to continue working with that?

My ffmpeg command line is: ffmpeg -rtsp_transport tcp -err_detect ignore_err -i rtsp://url -c:v libx264 -crf 21 -preset veryfast -g 25 -sc_threshold 0 -c:a aac -b:a 128k -ac 2 -f hls -hls_segment_type fmp4 -hls_time 4 -hls_flags independent_segments -hls_playlist_type event -hls_fmp4_init_filename 'init.mp4' -force_key_frames 00:00:00.000 stream.m3u8

[jnanes]

Really interested in this. Is there somewhere I can donate?

[Supereg]

@california444 sorry for the delay. Seems interesting, I'm trying to test it this week. If I'm finished with the hap-nodejs v0.8.0 beta update I hope I can get back to working on this. Maybe we can work on some sort of preliminary/beta API for secure video.

[california444]

HI,

in the meantime I (maybe) found an even better alternative, but hadn't time to test it, yet. This Project MP4Frag pipes data and fragment mp4 into an initialization segment and media segments. That sounds promising.

If I find time to test I will share my result again...

[california444]

HI,

my test result with MP4Frag is uploaded here.

That looks even better and is a lot more convenient. MP4Frag buffers the lastest x segments as well as the initialisation data for you. I configured the buffer size to two segments. After that I set up a callback for new segments as soon as they are completely available....

Generated files are:

• a full mp4 file (playable in Quicktime)
• a initialisation mp4 (contains ftyp and moof)
• an init + 2* 4s of buffer mp4
• 2 segments each appr. 4s long (moof and mdata, first frame is a keyframe)

Here is my ffmpeg command is used: const ffmpeg = spawn( 'ffmpeg', ['-reorder_queue_size', '5', '-rtsp_transport', 'tcp', '-i', 'rtsp://url', '-c:v', 'copy', '-preset', 'veryfast', '-force_key_frames', '0:00:04', '-sc_threshold', '0', '-c:a', 'aac', '-b:a', '128k', '-ac', '2', '-f', 'mp4','-frag_duration', '4000000', '-fflags', '+genpts', '-movflags', 'empty_moov+default_base_moof', '-metadata', 'title="Homekit Stream"', '-reset_timestamps', '1', 'pipe:1'], {stdio: ['ignore', 'pipe', 'inherit']} );

[Wildcat1]

All, just catching up with this thread and with many cams to test against, getting HSV running with home bridge camera-ffmpeg would be FANTASIC ! @donavanbecker @KhaosT just wondering what the latest is from your input WRT being able to implement this ? Thanks all, very cool if this can work

[Sunoo]

@Wildcat1 To my knowledge, nobody has quite figured out HSV yet (though it seems we’re getting pretty close). It does seem like I should be able to get support added to this plugin once people get those details ironed out though.

[Supereg]

@Wildcat1 don't know if you discovered the issue already over at the hap-nodejs repo. If you didn't could also be interesting to read in regards to HomeKit Secure Video https://github.com/homebridge/HAP-NodeJS/issues/741

[roysbike]

I have a xiaomi aqara G2H camera with HSV . how can she help? Ready to sniff answers from her or connect to her via serial

[california444]

@california444 sorry for the delay. Seems interesting, I'm trying to test it this week. If I'm finished with the hap-nodejs v0.8.0 beta update I hope I can get back to working on this. Maybe we can work on some sort of preliminary/beta API for secure video.

@Supereg are you generally working on HSV specific service implementations in HAP-NodeJS? I saw that some of the required services are not available, yet. Are you planning to integrate them in a future release?

[Supereg]

@california444 Once I'm through my exam phase and homebridge v1.2.0 (and hap-nodejs v0.8.0) is finished and released, my plan was to start working on a preliminary HSV implementation.

[bdruth]

@Supereg - do you have the code you hacked together to test out the HSV protocol pushed up anywhere? Might be helpful for some folks to get familiar with that so we can be in a better position to help?

[Finch106]

I purchased a few Eufy cameras this week that offer HomeKit/HSV so I can provide whatever you need as far as UI or whatever. As an aside, using HomeKit basically requires you to bypass all the native camera app features and use the camera only as RSTP it seems. If a camera uses AI to detect people vs pets, local recording, or if a camera has pan/tilt, those features are disabled unless you use the camera's native app.

[hazcod]

@Finch106 do you mean srtsp? (Secure RTSP) I suppose homekit secure video wouldn't stream plain RTSP over IP.

[llemtt]

homekit uses SRTP (Secure RTP) streaming, there's always a lot of confusion:

RTSP (and it's secure version SRTSP) it's an higher level streaming protocol that manages the "control plane" of the streaming (sessions setup, start/stop,...), it's the "streaming version" of HTTP. RTSP can support different "data plane" streaming protocols like RTP, TCP, HTTP, UDP,...

RTP/SRTP requires a separate "control plane" to work. Homekit "control plane" is embedded into HAP.

cheers

[rncry]

Following

[Wildcat1]

Question for all here. I'm running iOS 14 beta and tvOS beta. Even with the cams bridged through HomeBridge into HomeKit, I cannot display them on the latest tvOS beta. Anyone been able to test this yet ? Does it require HKSV to display on the APpleTV in tvOS beta. Trying to determine what the dependencies are in showing my cams that show fine in Home app (thanks to Homebridge + Camera-FFMPEG) but again NOT in the tvOS display camera area. Thanks

[Sunoo]

As far as I know, that shouldn’t require secure video. I honestly had completely forgotten that AppleTV could show cameras now, and never thought to test it. I do know the preview from the doorbell at least is working as expected.

[gondalez]

@Wildcat1 I have the latest beta and can confirm my cameras show in the home overlay (I'm using camera-ffmpeg also, not secure video).

Have you set your cameras as favourite in the home app on your phone? From memory only favourite cameras show on the ATV.

[iRayanKhan]

@Wildcat1 I have 4 Homebridge Cameras, and 1 HKSV Camera. The tvOS viewer seems to not work, and hasn't in any beta.

Look into the HomeCam app.

[BadgerBadgerAndFox]

Mixed bag here with ATV 4k and TVOS 14. Ring running through homebridge does popup in ATV when pressed. Home App in ATV displays all my cameras, some times they will work in liveview sometimes they wont, snapshot updates is also hit and miss, after they do start live feed returning to the tile view they do keep streaming, when they do start they also take an age to start. Full screen the resolution is not great. They are Unifi Protect cameras also integrated via Homebridge. I have a single old dlink cam configured directly in ffmpeg but its pretty old and the camera is failing, requiring frequent restarts, i'll see if i can get that up and report back. For the protect cams i currently prefer the Unifi native ATV app.... would be good to see them working especially if they can be integrated as HKSV for enhanced alerting.

[hazcod]

Anyone that looked further into this now iOS 14 is out?

[lukasa1993]

is video playback only posible with secure video?

[Sunoo]

Yes.

[hackwell]

https://github.com/Supereg/secure-video-specification

[erokoder]

Any news about HSV with ffmpeg homebridge?

[Sunoo]

This won’t be able to work until HAP-NodeJS adds support. As they have locked their issue pending new developments, I will follow their lead.

https://github.com/homebridge/HAP-NodeJS/issues/741#issuecomment-725154331

[Sunoo]

There is currently a PR to add Secure Video support to HAP-NodeJS. Once that gets accepted and support is added to Homebridge, I plan to add support to this plugin as well. Keeping this issue locked for now as I have no ETA on any of that.

https://github.com/homebridge/HAP-NodeJS/pull/904

[mrsbaaa]

Gents,

i am not sure if you figured out how to add secure videos support yet?

[hazcod]

@mrsbaaa It's merged in homebridge https://github.com/homebridge/homebridge/pull/3056

[Sunoo]

Yes, currently my plan is to spend this weekend getting this working on this plugin, provided that life doesn’t get in the way.

[martijndierckx]

@mrsbaaa It's merged in homebridge homebridge/homebridge#3056

Now that’s what I call good news!

[martijndierckx]

@Sunoo Since many camera's wont provide the needed i-frame every 4 seconds @ 1920x1080. Will we see a flag which will make ffmpeg force it, like it's being done in f.e. Scrypted?: https://github.com/koush/scrypted/blob/d0988023950fc36532b7e8d8e1f885470f7526b8/plugins/homekit/src/types/camera.ts#L56

I know this will result in extra CPU overhead, but it's better than not having HSV :-)

f.e. My camera's output 1920 x 1088, with an i-frame every second in H264+AAC (via NX Witness)

[Sunoo]

@martijndierckx I assume I’ll have something like that if it’s actually required. I’ll need to do a lot of testing, and it’s likely the first release with HSV support will not catch 100% of the edge cases.

Also, I’m assuming that using the copy vcodec will be even more fragile than it is today. I may have to have a setting to ignore that for HSV video.

[martijndierckx]

Let me know if I can help you test. I'm very eager to get it working here :-) I love NX Witness as an NVR, and frigate does a proper job of object-detection. But I miss the seamless integration in Homekit...

[Sunoo]

I will keep that offer in mind, thanks. I’ll have to see how things go, but I’m really hoping to push out at least a beta version this weekend.

[micheljansen]

Awesome. I'm also happy to help testing if that's useful.

[fate8383]

also ready to test to help any time.

[dswarne]

100% available for testing

[martijndierckx]

@Sunoo any luck on getting a beta out on the hksv branch?

[adx74]

Yes i'm also ready to test the hksv branch, i already have homebridge v1.4.0-beta.7 installed.

[burnbrigther]

Ready! Don't want to try camera.ui. Are we close to beta release?

[nkpm]

Ready for testing as well. homebridge v1.4.0-beta.7 running well.

I tried installing with npm install -g Sunoo/homebridge-camera-ffmpeg#hksv --save but no look yet.

Should it work already?

[Sunoo]

The HKSV branch is not in a working state quite yet. Sorry.

Also: For those who haven’t heard, I had a burst pipe in my house on Monday, which took out some of my cameras, the Raspberry Pis I use for development and testing, and my laptop power cord (among other things that don’t impact this). I’m hoping to get some sort of setup going soon so I can get back into it, hopefully this weekend, but things are a bit crazy over here right now.