av_interleaved_write_frame() error -32 while writing interleaved video frame

[GoogleCodeExporter]

In following the FFmpegFrameRecorder example from RecordActivity, I got this 
error when recording video frames:

av_interleaved_write_frame() error -32 while writing interleaved video frame

What does this error mean?

In recording each preview frame, I convert them to IplImage using this class I 
wrote:

public class IplImageBuilder {
    private opencv_core.CvMat nv21Image, rgbImage;

    public IplImageBuilder(int width, int height) {
        nv21Image = opencv_core.CvMat.create((int)(1.5 * height), width, opencv_core.CV_8UC1);
        rgbImage = opencv_core.CvMat.create(height, width, opencv_core.CV_8UC3);
    }

    public opencv_core.IplImage getIplImage(byte[] preview_frame) {
        nv21Image.getByteBuffer().put(preview_frame);
        opencv_imgproc.cvCvtColor(nv21Image, rgbImage, opencv_imgproc.CV_YUV2BGR_NV21);
        return rgbImage.asIplImage(); // TODO: does this IplImage need to be released?
    }
}

Original issue reported on code.google.com by eug...@wearableintelligence.com on 7 Feb 2014 at 9:13

[GoogleCodeExporter]

-32 might be this errno:
#define EPIPE           32      /* Broken pipe */
Which could be caused by a severed network connection, I guess... Does this 
happen when writing to files as well? 

And what happens if you run the code on the desktop with Java SE?

Thanks for any additional information you may provide!

Original comment by samuel.a...@gmail.com on 9 Feb 2014 at 1:52

[GoogleCodeExporter]

The network connection is open, though busy - the data from the audio loop gets 
through just fine. Haven't tried on the desktop yet. I am running this on an 
ICS Android, by the way

Original comment by eug...@wearableintelligence.com on 9 Feb 2014 at 5:25

[GoogleCodeExporter]

I have some more information. I switched to using the 
camera.setPreviewCallbackWithBuffer and now I get this error after about a 
dozen of "record(iplImageFromPreviewFrame)" calls rather than on the very first 
one

Original comment by eug...@wearableintelligence.com on 10 Feb 2014 at 10:05

[GoogleCodeExporter]

Here're my audio and video loops:

    private class AudioRecordRunnable implements Runnable {
        @Override
        public void run() {
            android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
            int channelCfg = numAudioChannels == 1? AudioFormat.CHANNEL_IN_MONO : AudioFormat.CHANNEL_IN_STEREO;
            final int minBufferSize = AudioRecord.getMinBufferSize(sampleAudioRateInHz, channelCfg, AudioFormat.ENCODING_PCM_16BIT);
            AudioRecord audioRecord = new AudioRecord(MediaRecorder.AudioSource.CAMCORDER, sampleAudioRateInHz, channelCfg, AudioFormat.ENCODING_PCM_16BIT, minBufferSize);
            short[] audioBuffer = new short[minBufferSize];
            audioRecord.startRecording();
            int bufferReadResult;
            while (isRunning && (bufferReadResult = audioRecord.read(audioBuffer, 0, audioBuffer.length)) > 0) {
                try {
                    recorder.record( ShortBuffer.wrap(audioBuffer, 0, bufferReadResult) );
                    Log.v(getClass().getSimpleName(), "Wrote " + bufferReadResult + " bytes of audio");
                } catch (Exception e) {
                    e.printStackTrace();
                    break;
                }
            }

            try {
                audioRecord.stop();
                audioRecord.release();
            } catch (Exception ignore) {
            }
        }
    }

    private class MyPreviewCallback implements Camera.PreviewCallback {
        @Override
        public void onPreviewFrame(byte[] data, Camera camera) {
            if (isRunning) {
                try {
                    maybeFixTimestamp();
                    recorder.record(iplImageBuilder.getIplImage(data));
                    Log.v(getClass().getSimpleName(), "Wrote " + data.length + " bytes of video");
                } catch (Exception e) {
                    handleError(e);
                }
            }
        }
    }

    private void maybeFixTimestamp() {
        long t = 1000 * (System.currentTimeMillis() - startTime);
        if (t > recorder.getTimestamp()) {
            recorder.setTimestamp(t);
        }
    }

My recorder is configured as follows (I'm publishing to an ffserver running 
with default cfg):
        recorder = new FFmpegFrameRecorder(publishToUrl, videoFrameWidth, videoFrameHeight, numAudioChannels);
        recorder.setFormat("ffm");
        recorder.setSampleRate(sampleAudioRateInHz);
        recorder.setFrameRate(videoFrameRate);
        recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
        recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);

Original comment by eug...@wearableintelligence.com on 10 Feb 2014 at 10:10

[GoogleCodeExporter]

Also, once this error happens, it continues to fire on every subsequent preview 
frame

Original comment by eug...@wearableintelligence.com on 10 Feb 2014 at 11:07

[GoogleCodeExporter]

So, this only happens over the network? Or with local files as well?

Original comment by samuel.a...@gmail.com on 11 Feb 2014 at 2:01

[GoogleCodeExporter]

Not with local files!

Original comment by eug...@wearableintelligence.com on 11 Feb 2014 at 2:07

[GoogleCodeExporter]

So it might a problem with the server. I'd try to make sure that the server 
works first.

Original comment by samuel.a...@gmail.com on 11 Feb 2014 at 2:11

[GoogleCodeExporter]

Well, like I said, I am using the standard ffserver configuration and posting 
to http://10.0.1.3:8090/feed1.ffm" (with recorder.setFormat("ffm")), so I'd 
expect that piece to work. The endpoint is certainly up and listening because 
my sound loop sends the data there just fine. Is it possible that the audio 
loop can be interrupting the video frames just because it has higher thread 
priority?

Original comment by eug...@wearableintelligence.com on 11 Feb 2014 at 2:17

[GoogleCodeExporter]

So, are you saying that it works when connecting with ffmpeg on the command 
line?

Of course you're going to need multiple streams if you're going to send from 
multiple different applications, if that's what you mean.

Original comment by samuel.a...@gmail.com on 11 Feb 2014 at 2:21

[GoogleCodeExporter]

It works when I give it a local file rather than a url.

I am sending both video and audio to the same url, though from different
threads

Original comment by eug...@wearableintelligence.com on 11 Feb 2014 at 2:28

[GoogleCodeExporter]

Yes, ok, that's fine, so, instead of your application, if you use ffmpeg on the 
command line, does it work, or not?

Original comment by samuel.a...@gmail.com on 11 Feb 2014 at 2:31

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

If I switch from using "ffm" format and posting to http://...../feed1.ffm
 to using "flv" and posting to "rtmp://....." (Wowza server was the
recipient), the transfer works. Posting a format=flv stream to a local
file works as well. By the way, when I say "works", I mean that data
gets continuously transferred and something starts happening on the
screen. This "something" is actually extremely slow, choppy, lacks
sound and is rendered in green shades.

I also did another experiment to answer your question about the command
line. I pre-recorded an flv on disk and then sent it to ffserver recipient
using this command:

ffmpeg -re -i myvideo.flv -acodec copy -vcodec copy
http://10.0.1.3:8090/feed1.ffm

That worked (I got the same unintelligible result).

So, it appears that I simply cannot send an 'ffm' stream. Sure, ffm is an
ffmpeg-only format, but one would think that if JavaCV includes FFMpeg,
that format should be supported as well. Thoughts?

Original comment by eug...@wearableintelligence.com on 11 Feb 2014 at 6:55

[GoogleCodeExporter]

Are you saying that ffmpeg on the command line, without JavaCV, gives poor 
result?

So, by your own admission, is the problem not with FFmpeg? And has nothing to 
do with JavaCV?

Original comment by samuel.a...@gmail.com on 12 Feb 2014 at 6:02

[GoogleCodeExporter]

Not quite. There are 2 problems here. One is the scrambled video. This
issue is not about that. This issue is about the other problem - the error
in transmission in a specific format.

Original comment by eug...@wearableintelligence.com on 12 Feb 2014 at 6:38

[GoogleCodeExporter]

Ok, so, what version of FFmpeg did you use on the command line? And which 
configure options did you use to build it?

Original comment by samuel.a...@gmail.com on 12 Feb 2014 at 6:50

[GoogleCodeExporter]

ffmpeg version 2.1.3 Copyright (c) 2000-2013 the FFmpeg developers

  built on Jan 17 2014 14:26:08 with Apple LLVM version 5.0
(clang-500.2.79) (based on LLVM 3.3svn)

  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter
--enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus
--enable-libtheora --enable-libschroedinger --enable-libopenjpeg
--enable-libmodplug --enable-libvpx --enable-libspeex --enable-libass
--enable-libbluray --enable-gnutls --enable-fontconfig --enable-libfreetype
--disable-indev=jack --disable-outdev=xv --mandir=/opt/local/share/man
--enable-shared --enable-pthreads --cc=/usr/bin/clang --arch=x86_64
--enable-yasm --enable-gpl --enable-postproc --enable-libx264
--enable-libxvid

  libavutil      52. 48.101 / 52. 48.101

  libavcodec     55. 39.101 / 55. 39.101

  libavformat    55. 19.104 / 55. 19.104

  libavdevice    55.  5.100 / 55.  5.100

  libavfilter     3. 90.100 /  3. 90.100

  libavresample   1.  1.  0 /  1.  1.  0

  libswscale      2.  5.101 /  2.  5.101

  libswresample   0. 17.104 /  0. 17.104

  libpostproc    52.  3.100 / 52.  3.100

Original comment by eug...@wearableintelligence.com on 12 Feb 2014 at 10:19

[GoogleCodeExporter]

Ok, thank you. Now we know it works fine with FFmpeg 2.1.3, so maybe they fixed 
something in that version. Let's try the same version with JavaCV and see what 
happens.

Original comment by samuel.a...@gmail.com on 13 Feb 2014 at 1:16

[GoogleCodeExporter]

The new release of JavaCV 0.8 now comes with FFmpeg 2.2.1, so if this is a 
problem with FFmpeg 2.1.1, this should resolve this issue. If it does not, 
please let me know.

A lot of other things have changed with this new version, so please make sure 
to read the news release here:
    http://bytedeco.org/release/2014/04/28/first-release.html

This new version also comes with easier to use build scripts, so please take 
advantage of that to try out different sets of options to satisfy your needs.

Thanks for testing this out for me!

Original comment by samuel.a...@gmail.com on 29 Apr 2014 at 1:31

• Changed state: Fixed