Live stream

[kloknibor]

Hi!

I would like to see an live stream! Octoprint is capable of that too so the raspberry pi should be able to handle that I guess... But the pro of it is that you can offer your client a livestream of his print (and even with DLP it's awesome to see for most) or you can use the livestream to see if the print still isn't failing ;)! (Underneath the VAT you can see if the parts are still sticking to the build platform instead of the VAT and if it's curing properly) this way I don't have to run upstairs 50 times during the whole print :)!

[WesGilster]

Looks like people are making use of MJPEG for live streaming, I think Ergobot mentioned this on a previous post. The format is trivial compared to the progressive download that I've put together for video recording. The fun part is going to be streaming to multiple live clients.

Maybe I'll have a live button on the video bootcard that disables the Video controls and streams through the poster. It'll be a pretty solid solution in a pretty quick amount of time without adding any new dependencies to the build.

[kloknibor]

I leave the coding side up to you! Maybe it's possible to restrict the streams to an x number of clients ;)!

To be fair I think it would be pretty cool to make a timelapse and stream at the same time ;)! Not sure if the raspberry pi can handle that tough...

[jmkao]

I'm not sure if the RPi camera has onboard h.264 encoding, but the Logitech C920 camera does, and thus allows you to do live streaming at extremely low CPU usage.

I use the following script to set up a streaming HTTP server with VLC:

#!/bin/bash

#VIDEODEV=/dev/video0
VIDEODEV=`ls /dev/video? | head -1`
XRES=320
YRES=180
FPS=10

v4l2-ctl --device=${VIDEODEV} --set-fmt-video=width=${XRES},height=${YRES},pixelformat=1
v4l2-ctl --device=${VIDEODEV} --set-parm=${FPS}

cvlc v4l2://${VIDEODEV}:chroma=h264:width=${XRES}:height=${YRES}:fps=${FPS} --sout '#standard{access=http,mux=asf,dst=192.168.10.26:8080,name=stream,mime=video/x-ms-asf}' -vvv

And can stream up to 1920x1080 at 30fps with < 5% CPU usage. To watch the stream, point VLC at http://:8080/

I think it should be possible to feed the correct cvlc commands to CWH as the streamingCommand property to allow for the current camera features to use any video4linux device compared to using raspvid, but I haven't played around with this too much.

However, as people have been noting in some of the other issues, if you combine video streaming with a wifi USB device, the RPI will become inaccessible. At 1080p, this occurs in < 5 minutes. At 320x180x10fps, this takes a few hours, but eventually the system will go down.

[WesGilster]

Yeah, I really hate saying it, but the Pi might not be the best platform for pushing large amounts of data over wifi. Again, the solution that the community is pushing is a script that bounces the interface on error, I suppose I'll write that as a "Feature" plugin. However, I don't really want to turn that feature on as a default option.

I've already coded the streaming solution I mentioned earlier in the thread. On the other hand, after looking at what you've presented, I think I should put together an alternative option as well... Pros and cons for my native java solution: Pros:

1. I still only need to open a single port for all CWH streaming and web communications.
2. When the time comes to secure these stream ports, it's will be very easy with a single annotation.
3. I don't need any further external dependencies.
4. I've implemented this with an optional CPU utilization governor to ensure that the Raspberry Pi(or any other hardware) will never get overwhelmed. Instead, streaming performance will only take a maximum of 1 CPU no matter the number of clients. Under this situation, the streaming content will simply degrade gracefully. Cons:
5. I haven't tested this, but I'm pretty sure my solution will have a higher upfront CPU cost than 5% of 1 CPU.
6. I have to maintain this solution (This is generally a pro, but in order to get maximum efficiency it could take more time than I'm interested in putting in to it.)

3rd party streaming tool integration: Pros:

1. They are proven and available now.
2. They are generally pretty easy to setup.
3. Apparently they are actually pretty efficient...
4. There is a community behind them. Cons:
5. I have to manage the installation and setup as a part of CWH.
6. I'm not really sure how to authorize and authenticate these streams without a proxy.
7. When things go south I have less control over the solution.

@jmkao It should be pretty straight forward to get those commands sent to the streaming command in CWH, but at the moment, that mechanism is only used for recording. As crazy as it seems, HTML5 video doesn't yet accept raw h.264 streams. Instead, the raw streams have to be encoded into a container first, I am using mp4parser to do this now.(Most people use ffmpeg) After I get the Java native solution working, I'll work on the direct stream of raw h.264 to the browser.

That will probably require one of the following:

1. some browser plugin
2. flash
3. some javascript streaming library
4. some sort of raw h.264 to MJPEG streaming library (this seems to be what Octoprint is doing)

If I've missed a simpler option, please clue me in...

[WesGilster]

I've implemented this in my development version and checked it in as: cwh226. I've also increased the the performance of raspistill by removing the default 5second timeout that is on it. I'm really not sure what that is for, but it was taking 5 seconds to run, and now it's taking milliseconds...

That was kindof a critical enhancement, because the new 'live' button is patching together images for it's live stream to inject into the mjpeg stream. I've also hardcoded a 1 second delay in between each frame because I wasn't sure what people's expectation are for streaming. I can easily correct this by adding a "framerate" text box on the Video bootcard to be used for three things: If recording:

1. If larger than 1 it will perform a normal recording.
2. If the number is between 1 and 0 it will perform a time lapse recording. If live streaming:
3. It will specify the amount of time between frames

I'm happy with this as a short term solution, but I'd like to hear more input from other people's expectation as they test this?

As I mentioned, if you really feel like you are going to have several clients attempting to stream, you can limit to a single CPU in the config.properties...

[WesGilster]

I found two other benefits to the way I'm performing the live stream:

1. You are still able to take still pictures during a live stream. I'm guessing that's a pretty rare feature. :)
2. A nice restful URL format: /services/media/startlivemjpegstream/Test%20Printer/clientid/0.7127316002734005/x/100/y/100?_=0.3150208240840584

Regardless, I want to hear honest feedback about what you guys think. No matter what, I'll open another feature request for "stream farming" to other apps and the ability to set FPS for time lapse.

[WesGilster]

I've also tried testing this with two streaming clients, and it works great. I'll also bet that it scales well from both a memory and CPU standpoint. The problems I'm seeing are:

1. The initial CPU costs seem higher than I expected.
2. The stream latency was higher than I expected (partially due to the 1 second delay I imposed, and partially due to continual native program startup costs.

Let me know if you guys do any scalability testing at different resolutions and different client counts.

[jmkao]

I think Octoprint runs mjpg-streamer as a separate process listening on a separate port, and then refers to it only in HTML to embed the streaming image.

I just updated my setup to include a powered USB hub that powers the Wifi, C920 USB camera, and the Pi itself that still is powered overall by one 2.5A USB power port. I'm running some stability tests now, but if all goes well, I'll do some testing with this using fswebcam or one of the other myriad generic v42l image capture tools.

[WesGilster]

Yep, I gathered the same thing about Octoprint. I've opened another feature #138 to hook into a "stream direct" solution just like they do. Then we'll need a way to tell the GUI what the new port and url will be.

That setup sounds perfect for testing out that wifi problem. I love the idea of testing with more software and getting more webcam hardware compatible. Thanks for the work...

[WesGilster]

Live stream has been implemented long ago and another bug #138 to integrate into other applications if the current live stream isn't sufficient.