getPngStream data emitter only fired twice

[andythecoderman]

When I call client.getPngStream().on('data', console.log); I only get two PNG's emitted each time I restart the client application. I've debugged far enough in to see that the underling video stream keeps emitting data but something is going wrong in the encoder and I haven't been able to figure it out yet.

Any ideas? https://github.com/andythecoderman/airbot/blob/master/video.js

[eschnou]

I just tried your code with a fresh install of the module (from npm) and it works great for me. I get hundreds of pngs. It disconnected once, but the reconnect code worked fine so the stream was interrupted for only 1~2 seconds.

So, some ideas on where the issue could come from:

• do you have the latest module version ? (0.1.0)
• what about your version of node ?
• are you sure nothing else is also connecting to the drone (e.g. your ipad/android navigation software)
• do you see error messages about disconnections?

[andythecoderman]

Thanks for checking it out, I'll give that a shot when I get home this evening.

I didn't get any errors, i'm using the ar-drone module from the master branch via npm link into my project.

Now that you ask, i'm not 100% sure what version of node i'm no now, I installed via apt-get a few days ago.

I don't think anything else is connecting, but will double check that my phone didn't try to auto connect on me.

I'm thinking it's something to do with ffmpeg and will dive in more from that side.

[eschnou]

Good point about ffmpeg. My ffmpeg version is 0.10.4, as for node I'm still in 0.8 (0.8.8).

[bencevans]

2 people including myself had this same problem at the Nodecopter Southampton event. Both using linux, node v0.8.22 & v0.10.15.

Also compiled ffmpeg to most recent:

ffmpeg version git-2013-08-10-b37ff48
built on Aug 10 2013 13:34:17 with gcc 4.7 (Ubuntu/Linaro 4.7.3-1ubuntu1)
configuration: --prefix=/home/bencevans/ffmpeg_build --extra-cflags=-I/home/bencevans/ffmpeg_build/include --extra-ldflags=-L/home/bencevans/ffmpeg_build/lib --bindir=/home/bencevans/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree --enable-x11grab
libavutil      52. 41.100 / 52. 41.100
libavcodec     55. 24.100 / 55. 24.100
libavformat    55. 13.102 / 55. 13.102
libavdevice    55.  3.100 / 55.  3.100
libavfilter     3. 82.100 /  3. 82.100
libswscale      2.  4.100 /  2.  4.100
libswresample   0. 17.103 /  0. 17.103
libpostproc    52.  3.100 / 52.  3.100

[rmehner]

We saw this too back in Florida. Most likely cause was wifi interference. Try to find an empty space for your drone (no other drones near) and try again. If it works, it's WiFi interference. A problem where we haven't found a good solution for so far, despite having more space.

[bencevans]

I had a bit of a play and found that the .getVideoStream() would work well when piped through the lib/video/PaVEParser.js parser would emit a working h264 stream that I could open with VLC or stream to the browser (only firefox seems to support h264 at the moment).

Thus I'm not ever so sure it is the WiFi as the PNG stream, at least I believe is a re-encoded stream derived from the .getVideoStream Parrot/h264 stream. I think the problem is somewhere in the decoding and re-encoding of the PNG frames.

[eschnou]

The decoding is indeed done by forking a ffmpeg process and I wonder if ffmpeg versions, or machine performance, could have an impact. The ideal would be to drop this ffmpeg hack altogether and find another way to decode the images. I wonder if some code of @bkw node-dronestream could be used for this.

[bkw]

Sorry for picking up this so late. The decoder of dronestream could certainly be run on the server, but it would only deliver YUV component bitmaps. You'd still have to convert these to rgb pngs somehow. That's why dronestream runs client side, since we can do all of that with a WebGL canvas in a very efficient way. Also we only have to transport the bare minimum to the browser.

Whenever I need pngs from the video, I use canvas.toDataURL() on the dronestream canvas and send the resulting uuenocded png back to the server.

[bencevans]

I can recommend LearnBoosts cairo backed server-side canvas module: https://github.com/LearnBoost/node-canvas

[bkw]

@bencevans last I time I checked, node-canvas did not support WebGL contexts.

[janpieper]

Changing ffmpeg to avconv in PngEncoder.js solved the problem for me.

ffmpeg version

$ ffmpeg -version
ffmpeg version 0.8.6-4:0.8.6-0ubuntu0.12.04.1, Copyright (c) 2000-2013 the Libav developers
  built on Apr  2 2013 17:02:36 with gcc 4.6.3
*** THIS PROGRAM IS DEPRECATED ***
This program is only provided for compatibility and will be removed in a future release. Please use avconv instead.
ffmpeg 0.8.6-4:0.8.6-0ubuntu0.12.04.1
libavutil    51. 22. 1 / 51. 22. 1
libavcodec   53. 35. 0 / 53. 35. 0
libavformat  53. 21. 1 / 53. 21. 1
libavdevice  53.  2. 0 / 53.  2. 0
libavfilter   2. 15. 0 /  2. 15. 0
libswscale    2.  1. 0 /  2.  1. 0
libpostproc  52.  0. 0 / 52.  0. 0

avconv version

$ avconv -version
avconv version 0.8.6-4:0.8.6-0ubuntu0.12.04.1, Copyright (c) 2000-2013 the Libav developers
  built on Apr  2 2013 17:02:36 with gcc 4.6.3
avconv 0.8.6-4:0.8.6-0ubuntu0.12.04.1
libavutil    51. 22. 1 / 51. 22. 1
libavcodec   53. 35. 0 / 53. 35. 0
libavformat  53. 21. 1 / 53. 21. 1
libavdevice  53.  2. 0 / 53.  2. 0
libavfilter   2. 15. 0 /  2. 15. 0
libswscale    2.  1. 0 /  2.  1. 0
libpostproc  52.  0. 0 / 52.  0. 0

[bkw]

Nice find! We should make this configurable and also try to find and use avconv by default, right? Am 21.10.2013 12:27 schrieb "Jan Pieper" notifications@github.com:

Changing ffmpeg to avconv in PngEncoder.js solved the problem for me. ffmpeg version

$ ffmpeg -version ffmpeg version 0.8.6-4:0.8.6-0ubuntu0.12.04.1, Copyright (c) 2000-2013 the Libav developers built on Apr 2 2013 17:02:36 with gcc 4.6.3 * THIS PROGRAM IS DEPRECATED * This program is only provided for compatibility and will be removed in a future release. Please use avconv instead. ffmpeg 0.8.6-4:0.8.6-0ubuntu0.12.04.1 libavutil 51. 22. 1 / 51. 22. 1 libavcodec 53. 35. 0 / 53. 35. 0 libavformat 53. 21. 1 / 53. 21. 1 libavdevice 53. 2. 0 / 53. 2. 0 libavfilter 2. 15. 0 / 2. 15. 0 libswscale 2. 1. 0 / 2. 1. 0 libpostproc 52. 0. 0 / 52. 0. 0

avconv version

$ avconv -version avconv version 0.8.6-4:0.8.6-0ubuntu0.12.04.1, Copyright (c) 2000-2013 the Libav developers built on Apr 2 2013 17:02:36 with gcc 4.6.3 avconv 0.8.6-4:0.8.6-0ubuntu0.12.04.1 libavutil 51. 22. 1 / 51. 22. 1 libavcodec 53. 35. 0 / 53. 35. 0 libavformat 53. 21. 1 / 53. 21. 1 libavdevice 53. 2. 0 / 53. 2. 0 libavfilter 2. 15. 0 / 2. 15. 0 libswscale 2. 1. 0 / 2. 1. 0 libpostproc 52. 0. 0 / 52. 0. 0

— Reply to this email directly or view it on GitHubhttps://github.com/felixge/node-ar-drone/issues/60#issuecomment-26706424 .

[janpieper]

I've modified PngEncoder.js to allow the user to change the video converter path via an client option: https://github.com/janpieper/node-ar-drone/tree/allow-different-converter

This solution is quite generic because it allows the user to specify any video converter, regardless if it is supported or not. I don't know if this is a good way to go. On the other hand, changing the converter path is required for users that does not have ffmpeg and/or avconv available in their PATH.

[felixge]

I've modified PngEncoder.js to allow the user to change the video converter path via an client option: https://github.com/janpieper/node-ar-drone/tree/allow-different-converter

I'm :+1: on making the converter configurable. That being said, your patch seems to assume that all converters take the same arguments? IMO the converter config needs to be a cmd template, or a callback function that returns the cmd. You're also assuming the converter has the same output, but that may be reasonable assumption for now.

[janpieper]

Yes, my implementation requires the use of ffmpeg or avconv. No other video converters are supported unless they use the same arguments but thats quite unlikely. There is no real difference to the actual implementation in master - master requires the use of ffmpeg :-) Now you are able to configure the path and the converter, but you are (still) limited to ffmpeg and avconv.

But yeah, we should find a more generic way to change and configure the converter to support all converters without having to change this module to support the converter.

[yocontra]

What about using node-which to figure out if they have avconv or ffmpeg and use the one they have? If they have both just use avconv since ffmpeg is deprecated