how can I get the h264 preview

[fengbenpaao]

hi,
I want to get the h264 preview; the MJPEG use the USB GPD is larget; if I can change to h264, can save resources； how can I do ?

[fengbenpaao]

ah, it is so good to finally show my h264 preview; but there are so much work to do,
there is only one step. go go go!!!!

[sappho192]

Please close this issue if you've solved it. And it would be nice if you leave a comment how you've solved it.

[fengbenpaao]

YES, I have solved it . but the video packet loss is serious;

frame->actual_bytes  is not same    frame->data_bytes;    

[elviswujian]

Hello，I am now doing the same work .I can add the h264 format in the project ,and can get the h264 frame.But similarly,the "frame->actual_bytes is always smaller than frame->data_bytes".That means the frame I got is incomplete. So the result of decoding with FFmpeg is unexpected.So,do you have some ideas to solve the problem.Maybe we can help each other.Thanks.

[fengbenpaao]

@elviswujian

hi, yes, I just use some callback fun to transfering frame for java ;
and use ffmpeg ( or MediaCodec ) to show;
but my picture is so bad: dalay and if I do moving, the pic will blurred ;

so I don't know lose packet ? or my ffmpeg issue ?

[fengbenpaao]

this is my doing: 1.set h264 frame to your camera to output: linux: $ lsusb -v VideoStreaming Interface Descriptor: bLength 28 bDescriptorType 36 bDescriptorSubtype 16 (FORMAT_FRAME_BASED) bFormatIndex 3 bNumFrameDescriptors 4 guidFormat {48323634-0000-1000-8000-00aa00389b71} and you can change : Stream.c

FMT(UVC_FRAME_FORMAT_BASED,
{'H', '2', '6', '4', 0x00, 0x00, 0x10, 0x00, 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}) //this values is 16 Libuvc.h UVC_FRAME_FORMAT_BASED ---> bFormatIndex = 3

2. get the frame for h264：

   UVCPreview.cpp: :do_preview()-->

   for ( ; LIKELY(isRunning()) ; ) { frame = waitPreviewFrame(); if (LIKELY(frame)) { do_h264_callback(frame);

//this you can callback the frame date for JAVA or other;

3. java and jni shara one buff memory: void UVCPreview::do_h264_callback (uvc_frame_t frame) { ENTER(); JavaVM vm = getVM(); JNIEnv *env; // attach to JavaVM vm->AttachCurrentThread(&env, NULL); if(!mH264CallbackObj){ LOGE("this is mH264CallbackObj is not null"); } if (LIKELY(frame)) {if(LIKELY(env)){ LOGE("this is do_h264_callback44###");

       }
   
       int buffSize=frame->width*frame->height*3/2;
   
       jobject buf = env->NewDirectByteBuffer(frame->data, buffSize);
   
         copy_frame(frame->data,frame->actual_bytes);
   
        env->CallVoidMethod(mH264CallbackObj, iH264callback_fields.onTest,frame->actual_bytes);      
             env->ExceptionClear();
    env->DeleteLocalRef(buf);

   EXIT(); }

4. then the java will get the h264 frame data; you can use ffmpeg or mediaCodec to decodes;

[jordanSunco]

@fengbenpaao nice! you did a great job! but i got the same problem as @elviswujian, how to fix it?

[elviswujian]

I have done the UVC project. The "frame->actual_bytes is no deed to same as frame->data_bytes".

[jordanSunco]

@elviswujian I know this, but why is the "frame->actual_bytes" may be 1/100 size of "frame->data_bytes", and the frame decoded actually is incomplete.

[fengbenpaao]

@jordanSunco yes, the h264 include I/P/B Frame, so frame->actual_bytes" < "frame->data_bytes"; so you must do same frame cache; It is not same Fmpeg;
besize, frame->data_bytes you can change ，reset calculation it;

[jordanSunco]

@fengbenpaao Thanks for your reply. Is there any code sample for decode UVC h264 payload?

[fengbenpaao]

@jordanSunco you can use the ijkplay to try, or decode frame to the yuv date tras.... for openGL, maby the videoplaying delay will be sovleted;

[elviswujian]

传到java层，使用MediaCodec。效果很完美。

[fengbenpaao]

@elviswujian qq多少，费那么多 口舌，哈哈

[elviswujian]

估计大家都是中国人，都用英语.......总之这个项目经历了很多波折。首先libuvc libusb在每个手机上面执行方式还不是一样的，有的手机是一帧一帧传输，有的是多帧组成一个数据块传输。较新的手机都是用的一帧一帧的传输，这其实涉及到的是手机内核的不同。同时也需要修改一些东西，不然传输h264数据有问题。传输这一块我是找了公司driver的同事一起搞的。然后解码可以使用ffmpeg，1K之内没问题。我现在是2K，解起来效率很低，就是用了硬解码。 就这样吧， 谢谢原作者，谢谢各位一起研究的同行。我的这个项目不会商用。

[jp1017]

@fengbenpaao perfect job, 太酷了,加我qq786241016学习下呗

同时, @elviswujian 大佬开源了啊,欣赏下呗, :smile: :+1:

[bsufo111]

@fengbenpaao Hello, 我目前也在做这块，但遇到一些问题没调通，加个qq吧，学习一下，qq523440831

[nwpuzc]

@fengbenpaao 你好，我最近有个项目正好用到这块，加个QQ请教一下，谢谢啦！我QQ是：554159078

[nwpuzc]

@elviswujian 你好，看到之前留言说你搞定了H264,并且开源了，但是现在看你的主页没有源码了，方便加QQ交流下么，最近有个项目用到这块，比较紧急，谢谢啦！我QQ是：554159078

[jiaxianhua]

this is my doing: 1.set h264 frame to your camera to output: linux: $ lsusb -v VideoStreaming Interface Descriptor: bLength 28 bDescriptorType 36 bDescriptorSubtype 16 (FORMAT_FRAME_BASED) bFormatIndex 3 bNumFrameDescriptors 4 guidFormat {48323634-0000-1000-8000-00aa00389b71} and you can change : Stream.c

FMT(UVC_FRAME_FORMAT_BASED, {'H', '2', '6', '4', 0x00, 0x00, 0x10, 0x00, 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}) //this values is 16 Libuvc.h UVC_FRAME_FORMAT_BASED ---> bFormatIndex = 3

1. get the frame for h264： UVCPreview.cpp: :do_preview()--> for ( ; LIKELY(isRunning()) ; ) { frame = waitPreviewFrame(); if (LIKELY(frame)) { do_h264_callback(frame);

//this you can callback the frame date for JAVA or other;

1. java and jni shara one buff memory: void UVCPreview::do_h264_callback (uvc_frame_t frame) { ENTER(); JavaVM vm = getVM(); JNIEnv *env; // attach to JavaVM vm->AttachCurrentThread(&env, NULL); if(!mH264CallbackObj){ LOGE("this is mH264CallbackObj is not null"); } if (LIKELY(frame)) {if(LIKELY(env)){ LOGE("this is do_h264_callback44###");

     }
   
     int buffSize=frame->width*frame->height*3/2;
   
      jobject buf = env->NewDirectByteBuffer(frame->data, buffSize);
   
       copy_frame(frame->data,frame->actual_bytes);
   
       env->CallVoidMethod(mH264CallbackObj, iH264callback_fields.onTest,frame->actual_bytes);      
            env->ExceptionClear();
   env->DeleteLocalRef(buf);

   EXIT(); }

2. then the java will get the h264 frame data; you can use ffmpeg or mediaCodec to decodes;

@fengbenpaao @elviswujian 请问可以留下联系方式吗？我现在需要处理 h.264 的，我的 qq: 501760984， 感谢！

[jinbo1996]

@fengbenpaao @elviswujian 大佬，听说你解决了H264，能否分享一下，感谢！我的QQ 1821066433

[jiaxianhua]

https://github.com/iOSDevLog/UVCCamera/tree/feature/surface_view USBCameraTest0, can preview h264, a lot of work to do. #463

[butterfly0Dream]

https://github.com/iOSDevLog/UVCCamera/tree/feature/surface_view USBCameraTest0, can preview h264, a lot of work to do. #463

I tried this project and can extract h264, but the data of the previous 10s cannot be decoded. I don't know what the problem is. Do you have this problem while using it?

[jinlongxu2018]

GOOD，我终于得到了H264视频，并且解决了前面一段时间H264解码后，花屏/绿屏问题。

[jinlongxu2018]

并且添加了UAC音频部分的支持,添加了设置不同的audio sample rate 功能等.

[jinlongxu2018]

https://github.com/iOSDevLog/UVCCamera/tree/feature/surface_view USBCameraTest0，可以预览h264，很多工作要做。第463章

我试过这个项目，可以提取h264，但是无法解码前10s的数据。我不知道是什么问题。你在使用的时候有没有遇到这个问题？

我已经解决了H264花屏问题!