search

WebThingsIO / onvif-adapter

ONVIF Profile S video camera adapter
Mozilla Public License 2.0
10 stars 6 forks source link

Lagging/ Intermittent Streaming #12

Open mhl787156 opened 5 years ago

mhl787156 commented 5 years ago

I'm testing this sytem with an ONVIF camera I bought off Amazon, almost all seems to be working (onvif discovery, connection, adding of the device). The only thing that isn't is that the streamed camera footage is very choppy and intermittent with buffering occurring every 2-30 seconds.

This occurs on both chrome and developer firefox. It does not appear to be a streaming issue as I have been getting a consistent 10-15fps when streaming using the ISpy desktop application. Logs dont seem to show anything of interest either.

The gateway is running on my home linux server running Ubuntu 16.04 with ffmpeg 4.1 installed. Do you have any thoughts on what could be causing this? Cheers!

mrstegeman commented 5 years ago

There are a couple things to note here.

  1. We're using DASH (or HLS) streaming. There tends to be a ~15-20 second lag automatically incurred here.
  2. We tried to use a reasonable framerate for streaming of 800 kbps, with the baseline H.264 profile, and 128 kbps AAC audio. This may still be too high for some connections.
  3. I'm not sure how iSpy works, but some things to keep in mind for the gateway:
    • If you're using the tunneling service, there is an extra hop and a bit of extra overhead between your browser and your gateway.
    • Your results will vary heavily based upon your internet connection's upload speed.

If you'd like to play with the ffmpeg options, you can do the following:

  1. SSH into your gateway (user "pi", password "raspberry")
  2. cd ~/.mozilla-iot/addons/onvif-adapter
  3. mkdir .git (this gets around checksum issues)
  4. Modify in lib/onvif-device.js. See code here: https://github.com/mozilla-iot/onvif-adapter/blob/master/lib/onvif-device.js#L242-L308
  5. In the UI, navigate to Settings -> Add-ons, then disable and re-enable (after a few seconds) the ONVIF add-on.

I would love to see performance tweaks if you can come up with some. I played around a lot with ffmpeg options and this seemed like a reasonable set.

mhl787156 commented 5 years ago

Thanks for your anwser! That's really useful to know and I'll have a go at playing around with ffmpeg and some settings later and let you know how I get on! As far as I understand, there's no tunnelling right now as its all on the local network.

A couple of things I noticed last night after posting was that the camera itself has a bunch of settings (bitrate, max frame rate, key frame interval, bit rate control) which i'll keep on tweaking, but it also has a second lower resolution stream (640x352) which might actually stream better than the high res stream (1920x1080). Perhaps it might be worth adding the option to switch streams on lower connections?

The other thing I noticed was that iSpy seems to be ~40 seconds behind which seems to me like they may be doing something more behind the scenes rather than just displaying the stream results, im not sure.

Ill have a play around and see what I can come up with!

On a different note, UI wise, Imo having a snap-shot button on the video streaming page would be a nice addition! Currently its hard to know what you are snap-shotting! Also in your installation instructions, its good to know that the default ffmpeg on apt-get is actually version 2.8, rather than ffmpeg 4/4.1 which is required on your project! It would be useful to make that clear for others possibly - that got me stuck for a bit on setup! :)

mrstegeman commented 5 years ago

It's been my experience that the ONVIF standard allows cameras to list a set of preconfigured profiles that can be selected. If I can figure out a good way to present those to the user, maybe I can add a "Profile" property. In the meantime, you could probably just set the camera to something lower and see if that helps your streaming performance.

As for ffmpeg, the add-on should detect versions properly. For instance, on my MacBook, I have version 4.1, but on the Raspberry Pi, I have 2.8. The add-on works properly on both, as it checks the version and only uses options which are valid for the installed version.

I like the idea of the snapshot button. I've also considered adding a snapshot action, which could be taken by a rule. However, we've been struggling to find a decent way to present all those snapshots to the user, and let the user manage them. Ideas?