search

pchpch11 / javacv

Automatically exported from code.google.com/p/javacv
GNU General Public License v2.0
0 stars 0 forks source link

FFmpegFrameRecorder has no support for audio #160

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
hi, samuel, 
    I am new to Android, android I want to use javacv/ffmpeg to encode camera data into .flv(h.264/aac), can javacv/ffmpeg be used in this way?
    Thank you very much!
What steps will reproduce the problem?
1.I have got data from Android camera(in onPreviewFrame(byte[] data, Camera 
camera))
2.I want to use JavaCV/FFmpeg to encode the data into flv(h.264/aac)
3.Can javacv/ffmpeg be used in this way?

What is the expected output? What do you see instead?
I just want to know that can javacv/ffmpeg be used in this way? And it would be 
better there is a demo, ha.

What version of the product are you using? On what operating system?
andriod 2.3, lenovo S2005A

Please provide any additional information below.
I would very much appreciate if you help me, thank you.

Original issue reported on code.google.com by zhangqia...@gmail.com on 27 Feb 2012 at 3:22

GoogleCodeExporter commented 9 years ago 
Sure it's possible. It requires some coding, you'll find what you need to know 
around the site and the mailing list, but I'll keep this issue opened as a RFE 
for a sample... Let me know if you would like to contribute your code if you 
get it working, thank you.

Original comment by samuel.a...@gmail.com on 27 Feb 2012 at 3:41

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
No, I think the easiest way would be to use FFmpegFrameRecorder. We can copy 
the YUV byte[] data into an IplImage, call cvCvtColor() on that, and write the 
output image using the recorder, e.g.:

FrameRecorder recorder = new FFmpegFrameRecorder("video.mp4", imageWidth, 
imageHeight); 
recorder.setCodecID(CODEC_ID_MPEG4);
recorder.setFormat("mp4");
recorder.setPixelFormat(PIX_FMT_YUV420P);
recorder.start();
// loop ...
cvCvtColor(yuvimage, bgrimage, CV_YUV420sp2BGR)
recorder.record(bgrimage);

I guess though if we are going to write YUV420P video data anyway, we should 
allow YUV images as input to FFmpegFrameRecorder and skip the RGB conversions 
... 

Or if you do no wish the OpenCV dependency, then you can always copy 
FFmpegFrameRecorder and hack your own mod. I am sure someone has done something 
like that. You should ask on the mailing list...

Original comment by samuel.a...@gmail.com on 28 Feb 2012 at 4:50

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Issue 168 has been merged into this issue.

Original comment by samuel.a...@gmail.com on 7 Mar 2012 at 5:55

GoogleCodeExporter commented 9 years ago 
BTW, I modified FFmpegFrameRecorder a bit to accept IplImage with nChannels == 
2 as having PIX_FMT_YUV420P .. This way, we can simply 
    IplImage.getByteBuffer().put(data)
    FFmpegFrameRecorder.record(IplImage) 
without having to do any conversion!

Not sure if it actually works, have not tested it, but would be great if you 
could, thank you!

Original comment by samuel.a...@gmail.com on 17 Mar 2012 at 5:34

Attachments:

GoogleCodeExporter commented 9 years ago 
Oh wait a minute, we should put that to PIX_FMT_NV21, but I'm sure you get the 
idea. Let me know if it works after changing PIX_FMT_YUV420P to PIX_FMT_NV21 
inside the attached file above, thanks

Original comment by samuel.a...@gmail.com on 17 Mar 2012 at 7:43

GoogleCodeExporter commented 9 years ago 
Am I possible to use this to merge images to a video?

Original comment by booker0108@gmail.com on 18 Mar 2012 at 12:15

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
No, look inside FFmpegFrameRecorder.java, search for PIX_FMT_YUV420P and 
replace that with PIX_FMT_NV21

Original comment by samuel.a...@gmail.com on 19 Mar 2012 at 9:22

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
It works, but it's slow.. It's faster to convert NV21 to YUV420P directly.

Original comment by samuel.a...@gmail.com on 19 Mar 2012 at 10:00

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Hi, samuel. now I am trying to transfer the recorded data with rtmp, but can 
not find the way, could you help me? thank you!

Original comment by zhangqia...@gmail.com on 21 Mar 2012 at 9:49

GoogleCodeExporter commented 9 years ago 
I know that someone on the mailing list has been working on that. You could ask 
there and see if you get any answers...

Original comment by samuel.a...@gmail.com on 22 Mar 2012 at 6:51

GoogleCodeExporter commented 9 years ago 
Thank god. I find the answer: if you have a rtmp server, you can easily do it 
with this code:
recorder = new 
FFmpegFrameRecorder("rtmp://192.168.1.27:1935/livestream/12070660", width, 
height);

Original comment by zhangqia...@gmail.com on 22 Mar 2012 at 9:16

GoogleCodeExporter commented 9 years ago 
now the important thing is how to add the audio, it seems that I should modify 
FFmpegFrameRecorder for audio recording, right?

Original comment by zhangqia...@gmail.com on 22 Mar 2012 at 9:20

GoogleCodeExporter commented 9 years ago 
Ah, for the client side, yes, but we cannot /send/ the data this way, and I 
have not coded a FFmpegFrameServer or something...

Original comment by samuel.a...@gmail.com on 22 Mar 2012 at 9:21

GoogleCodeExporter commented 9 years ago 
Yes, let me know if you add audio support and want to make a contribution..

Original comment by samuel.a...@gmail.com on 22 Mar 2012 at 9:22

GoogleCodeExporter commented 9 years ago 
but it does work, now I can play the stream on my computer.

Original comment by zhangqia...@gmail.com on 22 Mar 2012 at 9:22

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Wait a minute, I read this wrong. You could actually use FFmpegFrameRecorder to 
open a server socket?? Wow, interesting

Original comment by samuel.a...@gmail.com on 22 Mar 2012 at 9:24

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
yes, I just take place of "videoFile" with 
"rtmp://192.168.1.27:1935/livestream/12070660" in
recorder = new 
FFmpegFrameRecorder("rtmp://192.168.1.27:1935/livestream/12070660", width, 
height);

and it works, I also can't believe at first.

Original comment by zhangqia...@gmail.com on 22 Mar 2012 at 9:28

GoogleCodeExporter commented 9 years ago 
my Technical Director said that the ffmpeg includes a lib which can transfer 
stream with rtmp, rtsp, etc. but not a socket server.

Original comment by zhangqia...@gmail.com on 22 Mar 2012 at 9:35

GoogleCodeExporter commented 9 years ago 
Right, but it's still interesting! Thanks for letting us know.

BTW, the new FFmpegFrameRecorder I am attaching should work better at encoding 
NV21 data directly, but I haven't tested it. Let me know if it gives more 
interesting results than before.. Creating IplImage this way should work:
    IplImage image = IplImage.create(width, height, IPL_DEPTH_8U, 2);
    image.getByteBuffer().put(data)
    theFFmpegFrameRecorder.record(image) 
Let me know, thanks!

Original comment by samuel.a...@gmail.com on 22 Mar 2012 at 10:32

Attachments:

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Oh, I know how to solve it, just keep the value of Camera.Parameters.FrameRate 
same as the one of FFmpegFrameRecorder. eg:

        Camera.Parameters camParams = mCamera.getParameters();
        camParams.setPreviewFrameRate(20);
and:
        public VideoRecorder(String filename, int imageWidth, int imageHeight) {
        /* initialize libavcodec, and register all codecs and formats */
        av_register_all();

        this.frameRate   = 20; //30
    }

Original comment by zhangqia...@gmail.com on 23 Mar 2012 at 2:52

GoogleCodeExporter commented 9 years ago 
It works with raw data captured from the camera with Android? Hurray! :) Thanks 
for testing

Is the time OK if you call this before start()?
    theFFmpegFrameRecorder.setFrameRate(20);

Original comment by samuel.a...@gmail.com on 23 Mar 2012 at 4:49

GoogleCodeExporter commented 9 years ago 
it still seems a little fast, but very close.

Original comment by zhangqia...@gmail.com on 23 Mar 2012 at 5:56

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
setFrameRate() merely sets the frameRate field, just like you explain in 
comment #34, so if it produces different results, we've got a bug in Java 
here...

Yes, we need to modify the interface obviously

Original comment by samuel.a...@gmail.com on 23 Mar 2012 at 7:39

GoogleCodeExporter commented 9 years ago 
Hi,samuel,I rewrite the FrameRecorder.java and FFmpegFrameRecorder.java。After 

     recorder.setCodecAudioID(CODEC_ID_AAC);
I try to find audio encoder through:
     AVCodec codec_audio= avcodec_find_encoder(codecAudioID);
I got the error:encoder not find
Can you help me ?Thank you！

Original comment by fujiaoji...@gmail.com on 26 Mar 2012 at 3:53

Attachments:

GoogleCodeExporter commented 9 years ago 
I've added the new FFmpegFrameGrabber in the latest release, thanks for testing!

As for audio support, I am afraid I do not have any experience with that. 
Please open another issue about audio if this is an ongoing concern of yours. 
Maybe someone else who knows more will help out there... thanks!

Original comment by samuel.a...@gmail.com on 29 Mar 2012 at 12:41

GoogleCodeExporter commented 9 years ago 
Issue 181 has been merged into this issue.

Original comment by samuel.a...@gmail.com on 4 Apr 2012 at 1:33

GoogleCodeExporter commented 9 years ago 
Hi, samuel
     I set the recorder as: recorder.setCodecVideoID(CODEC_ID_H264); but
the LogCat said"codec not found", is CODEC_ID_H264 not included?

Original comment by zhangqia...@gmail.com on 8 Apr 2012 at 8:39

GoogleCodeExporter commented 9 years ago 
Nope, we would require libx264 for that, and I have not ported/compiled that :( 
Some people seem to have had success though, e.g.:
http://stackoverflow.com/questions/8812827/build-ffmpeg-with-x264-for-android

Original comment by samuel.a...@gmail.com on 8 Apr 2012 at 1:34

GoogleCodeExporter commented 9 years ago 
What version of FFmpeg and x264 should I use? (ffmpeg-0.7.11?)

Original comment by zhangqia...@gmail.com on 9 Apr 2012 at 2:33

GoogleCodeExporter commented 9 years ago 
FFmpeg 0.7.x yes, but as for the version of x264 that requires, you should 
check its documentation

Original comment by samuel.a...@gmail.com on 9 Apr 2012 at 1:42

GoogleCodeExporter commented 9 years ago 
I have finished compiling x264, but did not work, and my ffmpeg config.sh
like this:

#!/bin/bash

PREBUILT=/home/liang/android-ndk-r5/toolchains/arm-eabi-4.4.0/prebuilt/linux-x86
PLATFORM=/home/liang/android-ndk-r5/platforms/android-8/arch-arm

./configure --target-os=linux \
--arch=arm \
--enable-version3 \
--enable-gpl \
    --enable-libx264 \
--enable-nonfree \
--disable-stripping \
--disable-ffmpeg \
--disable-ffplay \
--disable-ffserver \
--disable-ffprobe \
--disable-encoders \
--disable-muxers \
--disable-devices \
--disable-protocols \
--enable-protocol=file \
--enable-avfilter \
--disable-network \
--disable-avdevice \
--enable-cross-compile \
--cc=$PREBUILT/bin/arm-eabi-gcc \
--cross-prefix=$PREBUILT/bin/arm-eabi- \
--nm=$PREBUILT/bin/arm-eabi-nm \
--extra-cflags="-fPIC -DANDROID" \
--disable-asm \
--enable-neon \
--enable-armv5te \
--extra-ldflags="-Wl,-T,$PREBUILT/arm-eabi/lib/ldscripts/armelf.x
-Wl,-rpath-link=$PLATFORM/usr/lib -L$PLATFORM/usr/lib -nostdlib
$PREBUILT/lib/gcc/arm-eabi/4.4.0/crtbegin.o
$PREBUILT/lib/gcc/arm-eabi/4.4.0/crtend.o -lc -lm -ldl" \
--extra-cflags="-I$PLATFORM/usr/include"

what config.sh should be like to match javacv?

Original comment by zhangqia...@gmail.com on 10 Apr 2012 at 10:11

GoogleCodeExporter commented 9 years ago 
Apply the patch in this package:
http://code.google.com/p/javacv/downloads/detail?name=ffmpeg-0.7.11-android-arm.
zip
That should do all that you need

Original comment by samuel.a...@gmail.com on 10 Apr 2012 at 1:00

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
a good news: it can load library already, but also a bad news, see the encoded 
file

Original comment by zhangqia...@gmail.com on 12 Apr 2012 at 8:14

Attachments:

GoogleCodeExporter commented 9 years ago 
Looks alright, just that your CPU seems too slow to encode in real time :) BTW, 
if you could provide the details of how you compiled and linked x264, it could 
help others who would like to do the same, thanks! And there seems to be audio 
as well, no? You figured this out too?

Original comment by samuel.a...@gmail.com on 15 Apr 2012 at 2:09