search

saki4510t / AudioVideoRecordingSample

Simultaneous audio and video recording sample using MediaCodec/MediaMuxer
Apache License 2.0
950 stars 291 forks source link

Adapt the encoding package to screen record #3

Closed RazorPT closed 9 years ago

RazorPT commented 9 years ago

Hi there

Your code its the first one i found which seems to working flawlessly while recording audio and video with MediaCodec, but i tried to convert the MediaVideoEncoder to capture screen and for that i used small bits of code from https://github.com/yrom/ScreenRecorder but when i try to record it will only record for a second and then stops no matter i long i leave it recording.

What i did was strip the RenderHandler from the MediaVideoEncoder since it doesnt seem i need it and pass MediaProjection object to the MediaVideoEncoder constructor and then i just add this to the end of MediaVideoEncoder prepare method:

mVirtualDisplay = mMediaProjection.createVirtualDisplay(TAG + "-display", mWidth, mHeight, mDpi, DisplayManager.VIRTUAL_DISPLAY_FLAG_PUBLIC, mSurface, null, null);

From what i understand the mSurface receives input from display after this method, it works and i capture the screen for just a 1 second then it stops any idea what i might be missing?

Thank you for your time

saki4510t commented 9 years ago

Hello,

I have not tested yet and this is just my supposition, but you mentioned important thing, it works and i capture the screen for just a 1 second then it stops.

This means no one call MediaEncoder#drain method and output buffer of MediaCodec encoder filled up. This is a reason why your code stopped after 1 second working.

As I know, unfortunately VirtualDisplay class (and MediaProjection class) have no callback method/listener to let us know when new frame is available. So I assume there are several ways.

  1. Call MediaEncoder#drain periodically according to your expected frame rate by yourself(using your private thread or handler).
  2. Call MediaEncoder#drain periodically using Choreographer and Choreographer.FrameCallback
  3. Create SurfaceTexture by yourself or get SurfaceTexture from TextureView. And add SurfaceTexture.OnFrameAvailableListener to accept notification event when new frame come and call MediaEncoder#drain from OnFrameAvailableListener#onFrameAvailable callback method. You can pass Surface created by new Surface(SurfaceTexture) to MediaProjection.createVirtualDisplay
  4. ...

I assume third one is the most efficient way.

saki4510t commented 9 years ago

Hello, There is another method if you use API>=19, you can use ImageReader class instead of SurfaceTexture. You can use ImageReader#getSurface to get Surface to pass MediaProjection.createVirtualDisplay. ImageReader has OnImageAvailableListener to let us know when new image is available and you can call MediaEncoder#drain from the callback method.

with best regards, saki

RazorPT commented 9 years ago

Ok just tried the following on the MediaVideoEncoder on prepare() method:

    ImageReader mImageReader = ImageReader.newInstance(mWidth, mHeight, PixelFormat.RGBA_8888, 2);

    mVirtualDisplay = mMediaProjection.createVirtualDisplay(TAG + "-display",
            mWidth, mHeight, mDpi, DisplayManager.VIRTUAL_DISPLAY_FLAG_PUBLIC,
            mImageReader.getSurface(), null, null);

    mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
        @Override
        public void onImageAvailable(ImageReader reader) {
            drain();
        }
    }, null);

But now i cant even record a second this is what i got on my logcat:

06-05 14:02:37.327 9372-9372/? D/MediaMuxerWrapper﹕ path=/storage/emulated/0/Movies/ScreenRecorderULTRA3 06-05 14:02:37.361 9372-9372/? W/VideoCapabilities﹕ Unrecognized profile 2130706433 for video/avc 06-05 14:02:37.384 9372-9372/? I/VideoCapabilities﹕ Unsupported profile 4 for video/mp4v-es 06-05 14:02:37.388 9372-9372/? W/VideoCapabilities﹕ Unrecognized profile 2130706433 for video/avc 06-05 14:02:37.389 9372-9372/? I/MediaVideoEncoder﹕ bitrate=12.36[Mbps] 06-05 14:02:37.392 9372-9443/? I/OMXClient﹕ Using client-side OMX mux. 06-05 14:02:37.465 9372-9443/? E/ACodec﹕ [OMX.qcom.video.encoder.avc] storeMetaDataInBuffers (output) failed w/ err -1010 06-05 14:02:37.466 9372-9443/? W/ACodec﹕ do not know color format 0x7fa30c04 = 2141391876 06-05 14:02:37.466 9372-9443/? W/ACodec﹕ do not know color format 0x7f000789 = 2130708361 06-05 14:02:37.470 9372-9443/? I/ACodec﹕ setupVideoEncoder succeeded 06-05 14:02:37.471 9372-9443/? W/ACodec﹕ do not know color format 0x7f000789 = 2130708361 06-05 14:02:37.709 9372-9390/? V/RenderScript﹕ Application requested CPU execution 06-05 14:02:37.713 9372-9390/? V/RenderScript﹕ 0xaec10200 Launching thread(s), CPUs 4 06-05 14:02:44.995 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.013 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.032 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.051 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.068 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.087 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.105 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.124 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.143 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.162 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.180 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.199 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.218 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.236 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.255 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.274 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.292 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.311 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.330 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.348 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.367 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.386 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.404 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.423 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.441 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.460 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.479 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.498 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.516 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.535 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.554 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.572 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.591 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.610 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.629 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.647 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.666 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.684 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.703 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.722 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.740 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.759 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.778 9372-9388/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.797 9372-9452/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned 06-05 14:02:45.815 9372-9389/? E/BufferQueueProducer﹕ [unnamed-9372-0] dequeueBuffer: BufferQueue has been abandoned

RazorPT commented 9 years ago

I noticed that with the new code i also cant now stop the muxer failed stopping muxer java.lang.IllegalStateException: Failed to stop the muxer at android.media.MediaMuxer.nativeStop(Native Method) at android.media.MediaMuxer.stop(MediaMuxer.java:225) at net.yrom.screenrecorder.encoder.MediaMuxerWrapper.stop(MediaMuxerWrapper.java:143) at net.yrom.screenrecorder.encoder.MediaEncoder.release(MediaEncoder.java:233) at net.yrom.screenrecorder.encoder.MediaVideoEncoder.release(MediaVideoEncoder.java:180) at net.yrom.screenrecorder.encoder.MediaEncoder.run(MediaEncoder.java:153) at java.lang.Thread.run(Thread.java:818)

The resulting video is a black window that says no video so the exception is thrown because there was no frame captured am i right?

RazorPT commented 9 years ago

Ok just tried with a simple thread that drains the encoder every 40ms to match the 25 frame per second and still i only got 1 second video?

saki4510t commented 9 years ago

Hello, I just add new repository ScreenRecordingSample

Sample app keep recording audio(from internal mic) and screen(using VirtualDisplay) using MediaCodec/MediaMuxer as .mp4 file until user explicitly stop recording even if Activity finishes. (app works as background service)

saki

RazorPT commented 9 years ago

Damn! Its working! What was i doing wrong? Will have to check your code and study it, anyway thank you so much :)

Also take a free time to try out Android Studio it really is superior to eclipse for android development :)