Add support for streaming video

goodrobots / MAVCesium

An experimental web based map display for MAVProxy based on Cesium

https://gitter.im/goodrobots/Lobby

GNU General Public License v3.0

41 stars 23 forks source link

Add support for streaming video #22

Open fnoop opened 7 years ago

fnoop commented 7 years ago

This would make an awesome FPV HUD if the map display was replaced by streaming video feed :)

SamuelDudley commented 7 years ago

advckpt01 My plan is to have an optional section that can be overlaid with FPV video. See the top middle screen in the attached image as an example. Is that sort of what you were thinking?

fnoop commented 7 years ago

Yes that's a nice idea. It would also be good to be able to replace the map display entirely with fpv, or at least a significant overlay. I'm thinking of mavcesium being served by a companion computer on a drone in the field, where it will rarely have internet access to download the tiles, and the realtime video would be more preferable to the rendered tiles.

SamuelDudley commented 7 years ago

I can put together a WIP to test this functionality. Do you think supporting live video from a gstreamer stream would be okay?

fnoop commented 7 years ago

Absolutely :)

see3peoh commented 7 years ago

This i have to try! That's an incredible idea. When the wip is avail I'm on board! Everything you've shown lately has been awesome!

tdrone993 commented 7 years ago

That would be great! I've modified mine a bit to have Nexrad Overlays, GPS Ground units (SAR) and 911 data even! Video integration would be great!

SamuelDudley commented 7 years ago

Sorry, I have been struggling with getting any free time to work on this recently... Hopefully this will change soon and let me take on a coupe of these features! This one is high on the TODO list.

SamuelDudley commented 7 years ago

Any thoughts on a low latency gstreamer to html5 pipeline? I have tried the following:

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! videoscale ! video/x-raw,width=320,height=240 ! theoraenc ! oggmux ! tcpserversink host=127.0.0.1 port=8080

and

<!DOCTYPE html>
<html>
        <head>
                <meta http-equiv="content-type" content="text/html; charset=utf-8">
                <title>gst-stream</title>
        </head>
        <body>
                <video width=320 height=240 autoplay>
                        <source src="http://127.0.0.1:8080">
                </video>
        </body>
</html>

In my testing the above gives about 2 seconds of delay between my webcam and the rendered video. Any thoughts on reducing that delay would be greatly appreciated! Cheers, Sam

tdrone993 commented 7 years ago

I looked into trying to sync the position and video maybe mjpg (supposedly it has timestamps embedded from the short research I quick looked into). ez-wifibroadcast has about 150ms range might see how they are tweaking some of the RPI camera settings.

poliant commented 7 years ago

why not overlaying directly on the video using picamera python library?

tdrone993 commented 7 years ago

http://www.linux-projects.org/2017/04/29/uv4l-webrtc-for-pizero/

tdrone993 commented 7 years ago

https://github.com/131/h264-live-player

SamuelDudley commented 7 years ago

Thanks for the links! :) I really need to spend some time and work out how these solutions might be able to be integrated into the existing python server setup. I guess first cut would be to try the supplied server and client to see how it really performs.

Syncing the video and the data is another issue but if I could embed a timestamp in with the h264 data frames it might should be able to keep it in sync...

SamuelDudley commented 6 years ago