Webcam stream (as option)

[Swiftnesses]

Hey @theyosh,

Loving this controller, mega work my friend - thank you.

I'm currently using MJPG Streamer (available here on Github) to stream my Duck COOP webcam, rather than use static images. On my Pi 3 (A+ with only 512MB RAM), I can stream and use the controller while only using around 15% CPU (which is amazing tbh!). I doubt this is even an effective method of streaming - likely modern browsers support much better alternatives (H264/265?).

I'd love to have the stream inside Terrarium Pi rather than running via another application. Could you please add streaming as an option, given the low CPU utilisation, it seems a real possibility which would hugely improve the options available to people wanting live updates (I need them, as there's a small chance a Duck could get caught in the door, sure this applies to reptiles too?).

Many thanks,

Mark

[Swiftnesses]

Seems like UV4L offers amazing streaming with low CPU usage and Pi cameras 👍

[theyosh]

The low cpu is due to low quality streaming. Both is motion JPEG And that is only supported in Firefox and Chrome. I think Safari can't handle it. So on iPhone it does not work well.. Also, the bitrate will be very high, and works well on a LAN. But going over the internet, you need a rather steady and fast upload speed.

So there is H264 and H265. H265 is not mature enough, so that is out of the picture. H264 is currently industry standard. But converting a stream to H264 cost a lot of CPU power. The Pi is able to do it... but will eat all CPU cores. So that is not an option.

I found a solution to use the Pi GPU for H264 encoding. You can mannully compile FFMPEG with --enable-mmal --enable-omx --enable-omx-rpi and then you can use the hardware encoder of the Pi. Which works pretty well. I have a Wifi webcam in a (birdhouse)[https://vogelhuisje.theyosh.nl/]. It is producing a multibitrate HLS stream. This takes about 1 CPU due to ffmpeg keyframe alignment and creating HLS chunks. But compiling ffmpeg takes about 1.5 hours. So not an option for normal users.

So it is a bit more complicated. And I do know, as my job is video transcoding and streaming for a company. So I have more then 10+ years experience in video encoding.

So, it is not that easy. Also both uv4l and mjpeg streamer have there own webserver and different ports that also needs to be forwarded.

And currently, I tile the webcam images for zooming like Google Maps. That is not possible with a live stream. And therefore the webcam controls should be changed...

What I can support is this: https://medium.com/@jaredtoddpetersen/home-security-live-streaming-with-raspberry-pi-f9293efca7ba

This will produce the same type of stream as my birdhouse. And that can be added as a webcam so that you have a livestream in my interface. But this will also need some more linux experience in compiling the software you selfs.

[theyosh]

Ah, all browsers support motion jpeg now a days

[theyosh]

Alternative: https://raspberrypi.stackexchange.com/questions/7446/how-can-i-stream-h-264-video-from-the-raspberry-pi-camera-module-via-a-web-serve

[Swiftnesses]

Thanks for the explanations, really helps me understand. Keeping it light, with as little extra bloat as possible would be the best solution, I'm streaming now at 1296x972 @ 30FPS in H264 on my Pi 3 and it's not using much CPU at all...

I'm not bad with Linux, but it would be awesome to have an option built in to activate streaming... the last link you posted sounds interested, fairly lightweight and compatible...

[theyosh]

How are you currently watching? Using vlc? Or using the WebRTC option?

[Swiftnesses]

Right now I'm using http://raspberrypi:8080/stream/video.h264

[theyosh]

that does not work in a browser right? That is playing in VLC?

[Swiftnesses]

Works in a browser ;-)

I'm using UV4L...

[theyosh]

Ok... cool... but I think I will try another way. I am currently compiling ffmpeg manually with the right codecs. And will test if that will produce HLS. That can be added to my software then...

But this will take some time to develop. And it will involve some manual work ;)

[theyosh]

And with an usb audio device and a mic you should be able to hear them also!

[Swiftnesses]

Hearing them would be very cool :-)

I'd like to add another camera in addition to the Pi V1 camera I use, any suggestions?

[theyosh]

Following: https://github.com/jaredpetersen/raspi-live#ffmpeg (so, manually compile ffmpeg)

created a ffmpeg with HLs support and H264 hardware. Now testing full Hd from the rpi cam to hls. cpu load 1% of 1 core :) This looks promising..

running:

raspivid -b 1000000 -o - -t 0 -w 1920 -h 1080 -fps 30 -g 30 | ./ffmpeg -nostdin -re -i - -vcodec copy -f hls -hls_time 1 -hls_list_size 3 -hls_delete_threshold 5 -hls_flags split_by_time+delete_segments+second_level_segment_index -strftime 1 -hls_segment_filename '/home/pi/TerrariumPI/static/hls/file%s-%%d.ts' /home/pi/TerrariumPI/static/hls/stream.m3u8

watch url: http://pi:8080/static/hls/stream.m3u8 (vlc or safari/iphone) Will add a hls player to the webcam part. This has only a 3 seconds delay :D

[Swiftnesses]

Nice! 👍

[Swiftnesses]

How did you get on, I'd like to try your stream method and check out utilisation....

[theyosh]

Ok, turns out... it is pretty easy to have a live stream now... Here we go!

• install ffmpeg: sudo apt install ffmpeg
• update my software: git pull
• restart my software: https://github.com/theyosh/TerrariumPI/wiki/FAQ#how-to-restart-terrariumpi
• clear browser cache
• add new webcam with location: rpicam_live

And that should give you a live webcam in the webcam sections. It is currently missing all zoom controls. Also archiving and rotation is not yet working.

And it is still alpha. But I got it working with 1.0% CPU usage :)

[Swiftnesses]

Testing now... :)

[Swiftnesses]

Okay, got the live stream... 2 issues I can see here:

1) My ducks are sideways, which looks uncomfortable (they need rotating!) :-) 2) I normally use 1296x972 on my V1 camera (Full FOV and fast FPS) - https://picamera.readthedocs.io/en/release-1.12/fov.html

For some reason changing the resolution has no effect, currently it's cropped their heads off...

Thanks for getting this working though, superb!

Edit: noticed you said rotation wasn't working yet, whoops.

[theyosh]

Yeah, that should be possible. But somehow my camera just stopped working ... :( And it is getting late here.. so tomorrow is another day.

[theyosh]

Ok, found the issue... it is getting late so stupid errors... Anyway, resolutions and rotations should work now

[theyosh]

Only the FPS is currently locked at 30 fps....

[Swiftnesses]

At this rate, my ducks will be getting more sleep than us :-)

I'll give it a test now and report back, get some sleep! :-)

[Swiftnesses]

To confirm, do I just do a pull, and then restart?

[Swiftnesses]

Hmmm, not working here, it now only plays for 5 seconds when I go to webcam (seems to be the same clip), no rotation or resolution change either?

[theyosh]

Just wait... do a new pull now. Just made an error.. and yesh. Do a git pull and restart.... that should do it

[Swiftnesses]

Did a pull, rotation / resolution still not playing here dude - sorry :-(

[theyosh]

hmmm... crap.. I have it working at the moment. Sorry for now... tomorrow I will try again

[Swiftnesses]

Hang on, one final cache clear and reboot of the app, and it's working! Good work.

[theyosh]

There is one downside on this technique. It will wear down the SD card. It writes 2 files every second... That has to move to a memory based storage... that will be next

[Swiftnesses]

[theyosh]

Cool! Need some at https://github.com/theyosh/TerrariumPI/issues/210

[Swiftnesses]

I'll take some pictures and post the setup, it's took a week - I nearly gave up, but we got there in the end (I've also been developing some IOT valves to use to automate their pond and water supply, extensive testing including freeze protection routines etc!).

[theyosh]

Ok, new update. This will write the files to memory so it will not hurt the SD card.

But you have to remove a folder by hand before restarting. So look at the location TerrariumPI/webcam/ There should be a folder like a94df035fc91a158c1f9d7a1c769ca66

That folder needs to be removed. It is possible to pull the new code first, and restart the software. During the 10 sec countdown, you have time to remove the webcam folder. Then it will be recreated with different settings.

Else you could get some errors.

[Swiftnesses]

Seems to work just fine ;-)

[theyosh]

Donald just woke up :) :D

[Swiftnesses]

It's crazy seeing them, one just tried to fly inside the coop, sometimes it's better not to see them ;-)

[Swiftnesses]

Now I just need to work out how to get the websockets working behind my proxy (nginx), it's serving okay, but switch statuses etc don't work when going via proxy pass.

Sorted it, just needed to add this to my proxy pass location (no need for NGINX on the Pi).

enables WS support

proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade";

[theyosh]

https://github.com/theyosh/TerrariumPI/blob/master/contrib/nginx.conf#L3-L6 and https://github.com/theyosh/TerrariumPI/blob/master/contrib/nginx.conf#L22+L23

[Swiftnesses]

Sorted it, see my post above ^^^

[ghost]

can I use a usb webcam for live camera?

[Swiftnesses]

Hey @theyosh this works great with the Pi Cam - but I seem to have a delay of 10 seconds, any idea why?

[theyosh]

@Swiftnesses the playlist consists of chunks of video of 2 seconds each. The list has 3 chunks, so 6 seconds of video. It could be that the player is having some trouble loading it, and will pause a bit. And that will increase the delay a bit. There is for about 12 seconds of video on disc. That is that the max delay. But 10 seconds is not a bad value.

@fenyx13 this technique is not suitable for usb or remote webcams. For that to work, a manually compiled ffmpeg is needed with special Pi H264 hardware support. That takes about 1 hour to compile and is rather technical. And then it 'could' work. There are more factors that can cause issues with USB or remote camera's. Also, that will also disable the use of the raspicam. You can either use the H264 hardware chip of the pi to watch H264 of create. But not both. So I will not make it for USB cams, as that is more complicated and goes beyond my support.

On the other hand, I am adding code to support HLS streams as webcams. Then there is a possibility to try for your self to install all the needed software. But I cannot give support on that.

Manual compiling according here is needed: https://github.com/jaredpetersen/raspi-live#ffmpeg

[Swiftnesses]

Hmmmm, the issue for me is that 10 seconds means I could have squished a duck before I realise :-)

When I tested UV4L it didn't have much of a lag, is this a ffmpeg quirk?

[theyosh]

Sorry, but if you need UV4L, you have to implement your self. Keep in mind, that I need to make software that works for everybody. And not to complicated. UV4l has its own webserver, which introduce new ports that needs to be opened on the router if you want to use it over the internet. So that is not an option.

HLS Technique: https://en.wikipedia.org/wiki/HTTP_Live_Streaming

I will never make the claim that my software is able to use as a kind of security system. It is a home automation system. For mission critical stuff is my software not suitable

[Swiftnesses]

@theyosh I can live with the 10 second delay if it can't be improved, the fact I can actually stream seamlessly is great, appreciate that!

[theyosh]

Ok. then we leave it as is... it should be max 6 seconds which I still think is pretty nice

[theyosh]

So, then this is fixed now? For non Raspicam devices this technique is not usable.

[Swiftnesses]

Unless you can reduce the lag, we can call it a success!

I have 2 cameras that offer static jpg, I'm hoping to add them to my duck run in various places, remote is supported right @theyosh ?

[Swiftnesses]

Also, I noticed there's a nice time overlay on iOS (but not on desktop), will you be adding archiving and overlays back?

[Swiftnesses]

🥇