Detecting if AppleTV or not...

[FrancoisDucobu]

Hello Everyone,

First, thanks for this plugin ! I have camera from HIK Vision and it works great.

In my home, I have 2 AppleTV and iPhone.

To avoid encoding issue, I used the option vcodec: copy The image was very good.

Then, when I tried on my AppleTV, it was not working... After searching on internet, I found that the apple tv is more picky with the homekits standards.

The Apple TV needs 802 kbps with 1920x1080. If it's not these values, it doesn't work... So, in order to get it work with AppleTV, I need to set the setting value empty to let the plugin encode the video with the right value.

But iPhone asks 1280 x 720 with 299 kbps :o So the quality is very bad...

I found a workaround but it's not very nicely done...

on the startStream method inside streamingDelegate.js I put these lines

let vcodec = null;
if(request.video.max_bit_rate !== 802){
    this.log.info("Not an AppleTV. Set vcodec to copy.");
    vcodec = 'copy';
}
else{
    this.log.info("AppleTV. We use the plugin config.");
    vcodec = this.videoConfig.vcodec || 'libx264';
}

    startStream(request, callback) {
        const sessionInfo = this.pendingSessions.get(request.sessionID);
        if (sessionInfo) {
            let vcodec = null;
            if(request.video.max_bit_rate !== 802){
                this.log.info("Not an AppleTV. Set vcodec to copy.");
                vcodec = 'copy';
            }
            else{
                this.log.info("AppleTV. We use the plugin config.");
                vcodec = this.videoConfig.vcodec || 'libx264';
            }
            this.log.info("vcodec: " + vcodec);
            ...

I'm not familiar with homebridge developpement and it is (for me) the only way to know if the request is from an AppleTV or from another devices.

Is there another way to do it ? Do you think you could add and option to use reencoding with AppleTV ? and "copy" or another option for others devices ?

Thanks a lot (sorry for my english)

Francois

[Sunoo]

The iPad Pro also requests 1920x1080 at 802kbps.

[the-robbie]

That workaround worked perfectly! Considering the best quality possible on an Apple TV or iPad is 1920x1080 at 802kbps, this ensures I get the best quality possible on all of my devices. The limitations on the Apple TV and iPad are annoying, but this makes it a bit more bearable as to not sacrifice quality on my iPhone and MacBook as well. Thanks so much for the tip, @FrancoisDucobu!

[Sunoo]

I’m toying with the idea of whether I want to implement this in the main plugin, and how I’d handle to config for it if I do.

It definitely seems to help resolve some end user issues, but it also feels like an incredibly fragile fix.

[Dunlapjc]

Thank you for all the work going into this.

I'd like to try this, but I need a little more insight. Can somebody point me to the correct folder for the StreamingDelegate.js file (I'm running homebridge with Homebridge UI installed).

Thanks, yeah, I get this is a fragile work-around...

I just can't get hardware encoding on my PI-4 to work worth anything @1920x1080. I know the PI-4 should be able to handle 1920x1080@30 FPS but it requires level 4 to be enabled... Can't figure that out either as that would likely be more solid/reliable than the patch above.

[Sunoo]

In my experience on the Pi 4 software encoding works better than hardware encoding.

[Dunlapjc]

Thanks Sunoo, I'll give it a go. My gut was thinking the other way, but I trust your expertise. My initial ties with software encoding at 1920x1080 were pixilated/blocky and FPS dropped to <15, so some tweaking to be done.

I can feed the stream from the camera at H264 or H264H at any FPS up to 30 with VBR or CBR and specify a max bit-rate (currently at 4096Kb/S).

If anybody has any feedback on settings to smooth it out and improve quality I'm all ears... (besides dumping resolution)

[Sunoo]

On my to do list is some real testing between the two, but I haven’t gotten around to it yet. The thing to keep in mind is the hardware codec block has never updated since the original Pi models, so what was a significant benefit on old, slow chips isn’t as great compared to the Pi 4.

[Kai9555]

Hello Everyone,

First, thanks for this plugin ! I have camera from HIK Vision and it works great.

In my home, I have 2 AppleTV and iPhone.

To avoid encoding issue, I used the option vcodec: copy The image was very good.

Then, when I tried on my AppleTV, it was not working... After searching on internet, I found that the apple tv is more picky with the homekits standards.

The Apple TV needs 802 kbps with 1920x1080. If it's not these values, it doesn't work... So, in order to get it work with AppleTV, I need to set the setting value empty to let the plugin encode the video with the right value.

But iPhone asks 1280 x 720 with 299 kbps :o So the quality is very bad...

I found a workaround but it's not very nicely done...

on the startStream method inside streamingDelegate.js I put these lines

let vcodec = null;
if(request.video.max_bit_rate !== 802){
    this.log.info("Not an AppleTV. Set vcodec to copy.");
    vcodec = 'copy';
}
else{
    this.log.info("AppleTV. We use the plugin config.");
    vcodec = this.videoConfig.vcodec || 'libx264';
}

    startStream(request, callback) {
        const sessionInfo = this.pendingSessions.get(request.sessionID);
        if (sessionInfo) {
            let vcodec = null;
            if(request.video.max_bit_rate !== 802){
                this.log.info("Not an AppleTV. Set vcodec to copy.");
                vcodec = 'copy';
            }
            else{
                this.log.info("AppleTV. We use the plugin config.");
                vcodec = this.videoConfig.vcodec || 'libx264';
            }
            this.log.info("vcodec: " + vcodec);
            ...

I'm not familiar with homebridge developpement and it is (for me) the only way to know if the request is from an AppleTV or from another devices.

Is there another way to do it ? Do you think you could add and option to use reencoding with AppleTV ? and "copy" or another option for others devices ?

Thanks a lot (sorry for my english)

Francois

Hi, I found the file and can change it. just I'm not a professional, so I wanted to ask if you can pull a larger code snippet so I can see where exactly the change needs to be added? Thx

[osis]

The Apple TV needs 802 kbps with 1920x1080. If it's not these values, it doesn't work...

Not sure if this helps, but I've been playing around with settings for awhile and can confirm this config works my iPhone X (IOS 15.1), 2020 iPad Pro (IOS 15.1), and my Apple TV 4K (TvOS 15.1.1).

My config running on the latest release (3.1.3 atm)...

"videoConfig": {
    "source": "-i rtsp://syno:pass@0.0.0.0:0/Sms=1.unicast",
    "maxWidth": 2688,
    "maxHeight": 1520,
    "maxFPS": 24,
    "maxBitrate": 3000,
    "vcodec": "copy",
    "audio": false,
    "debug": false
}

I run off of Synology's Surveillance Station so I'm already getting an h264 stream hence copy.

[Sunoo]

If you’re running copy, the max* fields are ignored. What’s the configuration of the stream on the Synology or camera side?

[osis]

Thanks for the tip! Will adjust my config.

On the Synology side things are pretty standard.

[Sunoo]

Interesting, you’re definitely above what AppleTV typically seems to want. I wonder if the newer tvOS versions have gotten more lenient with what they’ll accept.

[osis]

Before today I defaulted my config to this + copy which was okay but left me with connection times of 25+ seconds or timeouts.

Now that I've found the time to read some source code and docs to tinker. Learned maxStreams was defaulted to 2 and is the only max* var not affected by copy as well.

Updated config is this...

"unbridge": true,
"videoConfig": {
    "source": "-i rtsp://syno:pass@0.0.0.0:0/Sms=2.unicast",
    "maxStreams": 4,
    "vcodec": "copy",
    "audio": false,
    "debug": false
}

Tangible config diff between the original and new is...

• Added unbridge: true
• Removed -rtsp_transport tcp
• Removed -re
• Increased maxStreams (Lots of devices and NAS has tons of bandwidth. Using copy means streams have little to no impact on its CPU as well.)

Now I can see the same stream on all three devices and the stream consistently and reliably shows up in under 3 seconds 👏.

P.S. Thanks for all the hard work and let me know if you think adding a new config to the page with context would be helpful.

P.P.S. For fun, I tried with my Apple Watch (WatchOS 8.1) and all I got is Device appears to be inactive. Stopping stream. It did manage to connect once but it was going at 0.5 FPS. I'm guessing the quality of the steam overwhelms it.

[osis]

This was working great until the tvOS 15.5 update.

I can get stills but the stream is never loads.

[Sunoo]

I never watch my cameras on Apple TV, but I suppose it’s possible Apple has made changes. Can anyone else confirm whether it is still working on Apple TV? If not, I’ll look into it soon.

[fratinize]

I have 3 Apple TV's and at times one will show all my cameras they show LIVE, but some sometimes are not really live they appear stagnated. Most times I see only 1 or 2 camera the rest just flash something but remain "not connected"(no video)

[fratinize]

I did Noticed and this may be normal, but anytime I attempt to Show all 10 Cameras on my Apple TV 4K, My Pi 4 Heats up (Pushing 150f) and the CPU stays above 97% again only see 1 or 2 cameras, sometime they Drop. I wonder if it is too much for this Pi? I use the Defaults in the plugin Setup from the UI.

[Sunoo]

10 streams is an awful lot to expect from any Pi all at once.

[fratinize]

Hmm actually I have 14 1 is the ring, 4 are with UniFi protect and 9 are Wyze cams v2 through the camera ffmpeg plugin What would be a stable count? I don't object to adding another pi4/homebridge instance (that's possible?) may resolve some of the latency I experience with them. Thanx for the in-site.

[Sunoo]

I haven’t tested it, but I’d imagine something like four would be an absolute maximum? Video is hard.

[fratinize]

I Setup a second Homebridge on a 2014 MacBook Pro (High Sierra) Seemed to Have Alleviated some cpu stress on the pi. I can Also See all my Cameras on my Apple TV. They are At 1280/720 299 and seem to work fine. Only Challenge is when Audio is Selected, I can not connect to the cameras. I believe it has to dot with the Missing Codec on the High Sierra, having trouble finding it. See attached Photo of Apple TV. Shed Came has Audio Enabled.(this Also shows the same on my iPhone 13(15.6.1).

[sweetw0r]

For my Nanit Plus 720p camera I see no difference between "vcodec": "copy" and "vcodec": "libx264". "vcodec": "h264" however looks notably worse.

[2/25/2023, 10:04:22 PM] [Camera FFmpeg] [Nanit] Starting video stream: 1920 x 1080, 30 fps, 802 kbps
[2/25/2023, 10:05:53 PM] [Camera FFmpeg] [Nanit] Starting video stream: 1280 x 720, 30 fps, 299 kbps

Side note: Had to religion with my Apple ID on ATV for it to pick up the stream.