Intermittent camera stream hanging

[durosity]

I'm having an intermittent issue with the video streams of any camera stalling, particularly when viewing remotely. There doesn't seem to be any rhyme or reason to it, sometimes it'll stream successfully with no issues at all, close out and go back to the exact same camera and it'll start streaming for a few seconds, stall, then after a few more seconds i'll get the "No Response" warning. It happens with any of the 11 cameras i have setup. Closing the stream and going back in will immediately resolve it but it may recur or not. Also tapping details in the top corner and closing forces it to restart the stream. Are there any logs that could potentially shed some light on why its dropping?

[Ghawken]

Might be related to the Bitrate camera setting I found in my BlueIris testing. The default bitrate of 300k/s was to small and eventually would stall. Setting max bitrate to 1500 overcame this with the BI cameras. Might pay to do this for them all?

See my last 2 posts here: https://github.com/Colorado4Wheeler/HomeKit-Bridge/issues/66 Particularly bitrate one

Can turn on camera debug with videoConfig[‘debug’] = true setting then prints all ffmpeg stuff to log.

Might pay to add as setting?

[durosity]

Alas can't set that operator as i'm using this with SecuritySpy. However it may be a similar issue.. i limit the bitrate and fps from the camera itself to cut down server load, i might have to play with that.

[Ghawken]

Pretty sure you can set. It is a homekit-camera videoConfig setting. see the linked post, if feeling brave just add the videoConfig['maxBitrate'] = 1500 && or videoConfig['debug'] = True

in the securityspy videoConfig settings part, for example just around line 4450 in homekit plugin.py file. (in the space left for it!) Timezone catching up, good luck!

[durosity]

Ahh sorry I misunderstood, thought it was a BlueIris specific setting! I'll have a play with it this evening.

[Colorado4Wheeler]

Let me know if the bitrate thing works, I might be able to do something to let you config that in the plugin - although the whole point of integrating with SS and BI is so there is no additional config needed so I don't want to go overboard with "extra" settings.

[Colorado4Wheeler]

The next release I bumped the bitrate up to 1500. Just remember that by doing this you may experience severe delays when not in your local network. We'll have to play with it and see what the ramifications are and that will determine if there needs to be some kind of option.

[Colorado4Wheeler]

Ok, this is going to have to be an option I think because when I bumped mine up to 1500 my server couldn't keep up with the load and my camera feeds all got quite horrible. I went from having relatively perfect video to having super intermittent and choppy video. There's another setting I can put in there too for packet size and changing those two around might stabilize things but I'm not sure just statically setting 1500 as the bitrate is the answer - at least for SecuritySpy.

[durosity]

Might be worth just putting a couple of options in there for tweaking the packet size and bit rate and can have a play changing them about, see if we can find a combo that results in the most stable stream?

[Colorado4Wheeler]

Already WAY ahead of you, testing both of those settings in plugin prefs as I type this.

[durosity]

Test as you type? Multitasking.. impressive. I didn’t know you were a woman. 😂

[Colorado4Wheeler]

Things must compile and load young Padwan, these are the times I multi-task ;). Although studies show that multi-tasking is actually counterproductive and should be avoided and I can attest to that because each time things start to get really nuts (phones ringing, texts dinging, Git hub messages, etc) I get distracted and make dumb code mistakes.

[Colorado4Wheeler]

The new release is up, see how that works.

[durosity]

Yeah I really can't multitask. My brain queues people talking to me. I'll be typing something and my wife will ask me something, and only when i'm finished will I reply to her. Drives her nuts!

Hurm.. I've updated to the new release and all the cameras have disappeared from Home and can't seem to get them to re-appear.

[Colorado4Wheeler]

Not sure why that would be, I'm obviously using SS and the latest build myself and my cameras are running perfectly. Could be a HK glitch.

[Colorado4Wheeler]

Take a look at the config.json file, how does it look?

[durosity]

Hmm.. I've screwed this up somehow. Before you mentioned the config file I deleted the server device and created a new one and re-added the 11 cameras. 2 of the cameras i’ve added show up to add to Home but not the others. Config file has them all showing correctly. Same when I try to put them into another Home.

[durosity]

Aha.. It’d appear the cameras that aren’t showing up are ones that have the same name as the previous ones. The problem is i can’t see any way of removing them from the HK database.

[Colorado4Wheeler]

That happens with any accessory, it seems HK needs time to “purge”. I’ve run into that problem a half dozen times in testing, I’m still waiting for one of my servers to appear to get an idea of how long that process takes. I think you either have to delete the home it was in or wipe out HomeKit entirely - not sure yet. Hey, every princess has a wart or two :).

[durosity]

Ah yes, now I recall having that issue back when I was first playing around with HomeBridge and some basic URL commands for toggling some devices on and off. I just renamed them from Camera to Cam and they’re now all showing up correctly. Now to start seeing what bitrate, etc makes it work better. If only i could get a data connection at home..

[durosity]

Intermittently the cameras seem to be deleting themselves from the Home app. They show up ready to re-add but then just randomly disappear. I’ll have another play with them in the morning.

(Also I’ve noticed this happening with scenes disappearing after certain server items have been modified but not specifically relating to a device in that scene.. is that something you’ve come across?)

[Colorado4Wheeler]

Never had either happen. I’ve had servers blow up but never disappearing devices or scenes. I think yours might be possessed!

[durosity]

Clearly. I think the cameras was just a blip in homekit as when i went to re-add them it didn't even need the code. The Scenes and automations, however, after some fiddling, appear to happen when something is changed on the device server they relate to. I think i have a patern to replicate now so i'll try that when i get home and if i can i'll log it under a seperate issue.

As for cameras alas still no joy with various different bitrate settings and packet sizes. It's definetely less stable on WAN than LAN but both will drop eventually. I've noticed when on LAN it'll freeze for a while then stall but usually re-establish the feed before the No Response message appears, where as on WAN it'll freeze but then display no response. I've tried the 1500kbps option but that really didn't wanna work, especially on WAN.

@Ghawken Have you had any luck with any of the settings you've tried?

[Ghawken]

Been trying multiple, and I do mean multiple settings :)

Strikes me that BI could encode the stream correctly and then homebridge could just send the unaltered stream via ffmpeg copy command. BI does have a whole,lot of encoding options - but just can’t seem to get correct enought with options have. Have emailed developer and he may be happy to add a HomeKit compatible URL stream for direct copy. That should, maybe overcome any streaming issues. The BI app - plays the videos perfectly LAN, WAN wherever so must be multiple levels of conversion issue I think.

None of this is much help to you though, but in extensive altering settings did add a -maxrate to ffmpeg output Command. Seems to make a lot more stable on my testing - may be because limits the bit rate varability. More stable within both Lan and wan on my testing - might pay to give it a go?

Also you can un comment the log levelline - which gives a massive log output for ffmpeg should highlight any issue. Appears in the homebridge.log

This is ffmpeg.js lines 282:

        let ffmpegCommand = this.ffmpegSource + ' -map 0:0' +
          ' -vcodec ' + vcodec +
          ' -pix_fmt yuv420p' +
          ' -r ' + fps +
         // ' -loglevel 48' +
          ' -f rawvideo -tune zerolatency' +
          ' -vf scale=' + width + ':' + height +
          ' -b:v ' + vbitrate + 'k' +
          ' -bufsize ' +vbitrate +'k' +
          ' -maxrate ' + vbitrate +'k'+
          ' -payload_type 99' +
          ' -ssrc ' + videoSsrc +
          ' -f rtp' +
          ' -srtp_out_suite AES_CM_128_HMAC_SHA1_80' +
          ' -srtp_out_params ' + videoKey.toString('base64') +
          ' srtp://' + targetAddress + ':' + targetVideoPort +
          '?rtcpport=' + targetVideoPort +
          '&localrtcpport=' + targetVideoPort +
          '&pkt_size=' + packetsize;

[Colorado4Wheeler]

This is interesting because the default settings for HKB's cameras work perfectly for me on the LAN and the WAN, no issues whatsoever and it wasn't until I started changing those settings that things went south fast.

[Ghawken]

There are a lot of ffmpeg settings - but the maxrate option seems like it should be there, to me anyway. Certainly seems to have made a reasonable difference to my WAN and lesser LAN output. (these are 1920x1080, 15fps high bitrate streams it is converting)

see here: https://trac.ffmpeg.org/wiki/Limiting%20the%20output%20bitrate

The other aspect is why does the ffmpeg.js have the bufsize & b:v - all set to the bitrate of the file - looking at the vbitrate code I couldn't quite figure out why it was the same. It would seem ideally the bufsize should be 1/2 the bitrate. Might try that change as well.

My issue is the stream from the BI ioS app - is solid, smooth, bluray like depending on how low the bandwidth. The Homebridge camera stream not so much - but getting better! The problem is I can see it is possible with bandwidth, conversion etc - just how to get there!

As a separate aside the stream from temp. m - seems to be a mpegts packaged with x64 video within. Testing again the temp.h264 url stream - with more ffmpeg logging turned on - to see if can give answers as to why this stream doesn't work as well. (when you would think it should)

G

[Colorado4Wheeler]

My issue is the stream from the BI ioS app - is solid, smooth, bluray like depending on how low the bandwidth. The Homebridge camera stream not so much - but getting better!

@Ghawken you have to keep in mind that the HomeKit camera implementation is a supplemental nicety, and not the primary function of HomeKit. To me I don't need it to be high def Bluray quality streaming, I can always open up my SecuritySpy app on my AppleTV, iPad or iPhone for that if I want it.

If you can get it to that level that that's pretty cool indeed! For me, I'm happy with how it's working now - which is high definition, no fuss 300 bitrate and 30 fps.

I honestly have not had a single issue with my setup that I would consider out of the ordinary. I'm taxing my HomeKit pretty hard, quite hard actually, and it's running like a champ. I have no complaints - but I also have the bar set sort of low because so long as it responds to my voice commands the rest is all gravy.

[Ghawken]

You are right I am probably aiming too high!, and as you say the camera support supplement rather than main course.

That, main course, Homekit is indeed working very well, essentially I haven't had a problem with it. External access is also working without problem - something I always had difficult setting up before.

I can actually see it taking over a lot of current automations; hence my flurry of 'Issues' regarding sensors etc as getting sensor data with a voice platform has been something that has been lacking from Alexa implementation currently. (for obvious cloud authentication reasons)

Still - I'll kept seeing what options work better, and hoping to hear back from BI developer Ken; if he can get BI to produce a Homekit compatible stream that would perfect. (e.g /homekit.h264 equivalent). It seems like he is using the libx264 - so should be a matter of just having the same settings.

Test the -maxrate setting though - doesn't seem any downside - and certainly has improved video about 50%.

Glenn

[Colorado4Wheeler]

Test -maxrate where? On the source line? If it’s a compile option I likely won’t do that simply because the camera support is Homebridge-Camera-FFMPEG and when that gets updated then any compiled changes go away and I don’t want to worry about forking that at the moment, he’s done a real fine job with what is there. That being said, we may be able to enhance FFMPEG if it serves to improve the product markedly.

[Ghawken]

Just in the ffmpeg.js - javascript code that homebridge runs for camera support.

Basically it this homebridge plugin https://github.com/KhaosT/homebridge-camera-ffmpeg Still being actively changed, with a 'few' pending issues! And plenty of fine-tuning to come. (could focus some changes there I suppose?)

But it is Here: https://github.com/Colorado4Wheeler/HomeKit-Bridge/blob/master/EPS%20HomeKit%20Bridge.indigoPlugin/Contents/Server%20Plugin/bin/hb/node-v8.10.0-darwin-x64/lib/node_modules/homebridge-camera-ffmpeg/ffmpeg.js

Any changes are reflected immmediately on restart - and demonstrated in the homebridge log from the ffmpeg command run.

But you are right when the ffmpeg-camera plugin is updated will overwrite.
There have been some recent pull requests - particularly this one - https://github.com/KhaosT/homebridge-camera-ffmpeg/issues/164 referencing additional commandline options, and fixing a stderr issue causing stream to hang after 1-2 minutes. (May be relevant to the other issue): This addition

+        // Always setup hook on stderr. 
+        // Without this streaming stop withing one to two minutes.
+        ffmpeg.stderr.on('data', function(data) {
+          // Do not log to the console if debugging is turned off
+          if(this.debug){
+            console.log(data.toString());
+          }
+        });

But this is my:

This is my ffmpeg Command from line 282 downwards (should be obvious the change)

        let ffmpegCommand = this.ffmpegSource + ' -map 0:0' +
          ' -vcodec ' + vcodec +
          ' -pix_fmt yuv420p' +
          ' -r ' + fps +
         // ' -loglevel 48' +
          ' -f rawvideo -tune zerolatency' +
          ' -vf scale=' + width + ':' + height +
          ' -b:v ' + vbitrate + 'k' +
          ' -bufsize ' +vbitrate +'k' +
          ' -maxrate ' + vbitrate +'k'+
          ' -payload_type 99' +
          ' -ssrc ' + videoSsrc +
          ' -f rtp' +
          ' -srtp_out_suite AES_CM_128_HMAC_SHA1_80' +
          ' -srtp_out_params ' + videoKey.toString('base64') +
          ' srtp://' + targetAddress + ':' + targetVideoPort +
          '?rtcpport=' + targetVideoPort +
          '&localrtcpport=' + targetVideoPort +
          '&pkt_size=' + packetsize;

Glenn

[Ghawken]

Ok Not sure that this is best place to discuss - but I have done it! (not sure the usefulness however)

Well my that I mean proven that can get the same quality from HomeKit camera than can from native app.

I have changed camera-ffmpeg so that is doesn’t reencode simply copy the current 264 stream and then add audio from a secondary stream. This needed some changes bc the h264 stream BI produces is a ‘elementary stream’ which means no time code, no FPS possible etc.etc. Removing a few of these checks allows HomeKit to pass this stream unaltered to the iOS viewer. Depending on the bitrate selected and internet connection there is essentially perfect connection (can’t tell the difference from BI app). Without audio it is relatively straightforward.

Is this of any use? Maybe. The main difference being the Native apps know whether on LAN or WAN and use different connection sertings for each. We don’t have the same option on HomeKit, particularly if not rencoding. It may be possible to add that to the JavaScript and self select a stream, but probably not easily/automatically/foolproof. Sound - isn’t included in the raw h264, to get that I open a second stream and mux the audio in - which is working fine - but does add a level of complexity - probably meaning a fork.

Whilst you could get the plugin to send the whole ffmpeg command, rather than the ffmpeg.js adding it together. Then you could change it automatically for different cameras devices as needed (using the wiki as a guide) - this is probably a bit much a bit far. Particularly for a supplement to the main purpose of the plugin!

None-the-less it does work and can get flawless stream from within HomeKit; just not super easily without some work.

Glenn

[Colorado4Wheeler]

Well, you certainly are passionate about getting that video perfect in HomeKit! Everyone needs a hobby :). You know, last night I had cause to watch my stream for a while and it timed out after 3 minutes and I simply hit done and tapped it again for a another 3 and that didn’t bother me because it made me realize that, for me, I just want to see what’s happening this moment more than I want to actually watch it happening over time.

But if these changes make your HA integration stronger and better then it’s what you should do. I might recommend that you just grab a copy of the entire HB folder to merge into new releases and just keep an eye on release notes for when other things change. I actually do manipulate some of the Homebridge files away from the distro, but it’s not recompiling or digging deep into the JS, just a few tweaks here and there.

[Ghawken]

Will do. With further testing I am even more impressed with how good this stream is - testing today further away from LAN and it’s as good as native app. (which it should be as is not re-encoding)

Will fork and update fork with these changes for review.

Probably be useful to incorporate some of these changes particularly for BI users. The BI temp.m file is a mpegts with the x64 video and AAC audio. The other streams temp.h264 is the raw h264 stream without audio. Converting the mpegts x264 add issues I think. Changing to that stream for a camera without audio would probably give better result for all - though probably needs ffmpeg,js changes given the elementary stream nature of it. Although may be able to include this in the ffmpegsource just for BI cameras within HomeKit.

I see if I can work up a part solution without changing the ffmpeg.js file for the majority of BI users.

Glenn

[Ghawken]

Along these lines - does HomeKit give any indication whether connecting via LAN or WAN? (and you haven’t added any checks for this?)

If knew could potentially pull different streams once wireshark lets me know what they are when I’m back on LAN

Glenn

[Colorado4Wheeler]

I'm sure HomeKit knows but the question is if it tells Homebridge and then if we can access that information. @webdeck might know the answer to this one.

[webdeck]

I don't know. The best place to ask is on https://github.com/KhaosT/homebridge-camera-ffmpeg since the author of that plugin is also the author of homebridge.

[Ghawken]

Thanks - Think it might be a mute point.

Have BI video feed working really well; without audio it is probably doable within plugin code selector (leaving ffmpeg.js untouched - actually more testing reveals hard to not change ffmpeg.js code). Have to test this properly though. With audio probably not without changing both, and adding complexity to have compatible with SS as well. (although quite possible SS might work better without re-encoding as well but hard to test) Pretty much all my cameras have audio - but video is so much better; so personally probably worth the sacrifice, but probably need a straw-pole? Fantastic video, or not-so-fantastic but maybe some out-of-sync wrong samplerate audio with it?

Glenn

[Ghawken]

Congrats on ongoing progress.

Sorry about the wrong direction pull request! (oops)

I have HomeKit pulling the raw h264 stream unchanged - this works really well. I have added a most appropriate stream selector within the HomeKit-ffmpeg code. If BI will pick the nearest stream in bandwidth. This gives a very high quality flawless stream with video straight from BI (just copy)

Audio is an issue. It appears HomeKit cannot deal with a raw h264 video stream with audio - starts & get a few seconds of sound - then few seconds 64-65 secs later. I believe it is because the video h264 is missing any timesstamps - and can’t add them without making stream a lot worse. (By converting and defeating purpose)

Have spoken to BI developer and he is keen to add compatible rtsp stream to BlueIris sometime this year. That will probably overcome the audio issues and need another change down the track.

Glenn

[farberm]

I am using security spy and when I first setup the cameras I can add then and get an image. However streaming never really occurs and the orgimal image never changes for days. I have the FPS set formy cameras at 3-5 fps range. I have the Camera bitrate for the plugin at 1500 and the packet size at 1316 and the enhanced post at 8558. Do I have my setting incorrect?