getPngStream gets slower quickly

felixge / node-ar-drone

A node.js client for controlling Parrot AR Drone 2.0 quad-copters.

http://nodecopter.com/

MIT License

1.76k stars 428 forks source link

getPngStream gets slower quickly #66

Open flockonus opened 11 years ago

flockonus commented 11 years ago

On the first seconds the png stream is real-time, like a video, but then it gets a consistent ~1s delay between.

I am wondering if Node's buffers / pipes get slammed?

felixge commented 11 years ago

I'm not entirely sure, but IIRC I suspected FFMPEG to being doing something funky while developing this. To really find out one would probably have to use the ffmpeg C API to get more control / details on what is happening.

flockonus commented 11 years ago

Thanks man, I've done some test to discard some frames on https://github.com/felixge/node-ar-drone/blob/master/lib/video/PngSplitter.js#L55 but it didn't help at all, so it is consistent with FFMPEG getting slower

lord commented 10 years ago

This is a big problem I'm running into — it's hard to program the drone with OpenCV if the PNG files I'm sending to OpenCV are from several seconds ago.

felixge commented 10 years ago

I'm currently implementing this again myself (but unfortunately in Go) and have learned the following since doing the first implementation here:

The AR Drone Developer Guide PDF mentions a latency reduction mechanism that should be implemented by any receiver. The basic idea is to buffer all frames that are received while the encoder is busy, and when the encoder is available again, give it the latest I-Frame on the buffer, otherwise the next P-Frame. [1]
Directly accessing the ffmpeg c API for the decoding isn't as messy as I'd have expected, and can deliver much better results than the crazy child process thing currently found in this library.

So if somebody wants to go ahead and try to make a better video decoder for this module, that'd be great. I might end up doing it myself as we'll need it for NodeCopter at JSConf.us.

lord commented 10 years ago

Hmm, I'm not the best at Node, but I'll give it a try, since I definitely need it for my project. Thanks for the pointers.

lord commented 10 years ago

@felixge Working on this now. Quick question — it looks like the PaVE frames are being fed directly into FFMPEG, and not going through PaVEParser at all...can FFMPEG parse PaVE frames?

felixge commented 10 years ago

@lord FFMPEG has no concept of PaVE frames, but the h264 NAL units have a fixed signature (0x00000001 IIRC) which ffmpeg is able to find in the stream and ignore any "pave junk" in between.

lord commented 10 years ago

Ahhhh okay, that explains it, thanks!

zergler commented 9 years ago

Was anyone able to get this working? I have a pretty fast frame rate but there is a consistent delay of about 1 to 1.5 seconds.