search

hybridgroup / node-bebop

A Node.js client for controlling Parrot Bebop & Bebop2 quadcopters.
http://nodebebop.com
MIT License
146 stars 61 forks source link

Video stream issues: Jumpy MJPEG stream / isolating H264 frames from the stream #39

Open wilpulling opened 8 years ago

wilpulling commented 8 years ago

Hi,

I'm trying to do image processing on the video stream (from a Bebop 2) outside of node (for example, in a Python process). My goal is to use a named pipe to send each frame to the other process. I've managed to achieve this with getMjpegStream, but that stream is very inconsistent (even without my sending images over the pipe) and gives me about 1 fps. When I run the example, I get the following error repeatedly:

Error: Error loading file at Error (native) at Timeout._repeat (/home/pi/roboTC/controller_files/node-bebop/examples/mjpeg-stream.js:25:8) at Timeout.wrapper [as _onTimeout] (timers.js:417:11) at tryOnTimeout (timers.js:224:11) at Timer.listOnTimeout (timers.js:198:5)

And occasionally, I get the following error: Corrupt JPEG data: premature end of data segment

I'd either like to either figure out a way to improve the quality of the MJPEG stream, or figure out how to use the H264 stream. The example provided for using the H264 stream pipes the feed to a file, but I need to process in real time.

Can anyone recommend a fix for the MJPEG stream, or give me guidance / an example of how I can isolate H264 frames so I can convert them to Base64 and pipe them out?

Relevant previous threads:

9 (Live stream to browser) / #14 (newbie question. how can i get videostream off the bebop)

Both of these are focused on using Broadway to display the feed in a browser. Since I'm keeping the feed local it seems like this doesn't apply to my situation, and I haven't been able to get anything working after playing with Broadway.

Hardware and OS: Raspberry Pi 3 running Raspbian Jessie.

Thanks everyone, appreciate any help you can provide!

wilpulling commented 8 years ago

Some additional information: The "error loading file" I'm getting from mjpeg-stream.js is occuring every time the mjpg.on("data") listener receives a chunk which is less than ~40k bytes in length. These 'short' chunks are typically ~14k bytes in length. This makes me think the stream is emitting a partial frame.

Wanted to add this detail in case it gives anyone an idea of what's going on. Would still appreciate tips on using the h264 stream if that could be faster! Thanks!

deadprogram commented 7 years ago

Hi @wilpulling sorry about the greatly delayed response.

Please checkout the latest changes to the https://github.com/hybridgroup/node-bebop/tree/feature/firmware-4.0 branch I've added an example that uses RTP video:

https://github.com/hybridgroup/node-bebop/blob/feature/firmware-4.0/examples/rtp_video.js

Basically, you just turn on the video streaming on the drone during the initial discovery process, then some other code can connect to the RTP stream and control ports. This will work great from other language implementations in the same fashion, I've already added the same to Gobot: https://github.com/hybridgroup/gobot/commit/e828f5617dfa52c73dc771f5494caab20f78a409

Hope that helps!

tomrunia commented 7 years ago

Hi @wilpulling, did you managed to read the video stream in Python? I am trying to use OpenCV (3.0 Python) to read the video stream, but so far I have no luck.