Need to get the YUV frames.

[himanshu131098]

I see that in the initPacketizer function, a code block is written to get YUV frames from the mCodecManager:

mCodecManager.enabledYuvData(true);
        mCodecManager.setYuvDataCallback((mediaFormat, byteBuffer, i, i1, i2) -> {
            mTranscodeOutputSurface.awaitNewImage();
            mTranscodeOutputSurface.drawImage(true);
            return;
        });

I am able to get the YUV frames from it however the video vanishes from the mobile screen.

What can be the probable cause for this?

Thank You

Himanshu V Vyorius Drones New Delhi, India

[m4xw]

Hi, so theres a issue in the DJI SDK (at least when I wrote it), VideoFeeder provideTranscodedVideoFeed would not function if the DJICodecManager isnt created (even tho its not used!!) This is required for DJI to insert keyframes into the stream of data. Manually orchestrating it as per documentation just wouldn't work :/ (for Mini 2, I know it works for some drones)

Theres several DJI bugs at play. So when you setYuvDataCallback, its basically stealing images by overwriting the callback.

See VideoService shenanigans for further information:

    private void initVideoStreamDecoder() {
        DJIVideoStreamDecoder.getInstance().setYuvDataListener((mediaFormat, byteBuffer, size, w, h) -> {
            this.onDataRecv(byteBuffer.array(), size, 0, false, 0, 0);
        });
        //NativeHelper.getInstance().setDataListener(this);
    }

There I basically abused the existing interface to have VideoFeeder feed into the ffmpeg decoder (kinda double transcoded in feeder but ffmpeg is far more stable at cost of some perf!). DJIVideoStreamDecoder processes the data, does the native output in-app via mediacodec and feeds another copy of the data via the abused YuvDataListener to the the VideoService for GroundControl

[himanshu131098]

So is it possible to convert the byteBuffer from the setYUVDataListener into an OpenCV Matrix?

[m4xw]

You can just make some adjustments in MainActivity -> CodecOutputSurface, it also has a saveImage function, tho i never tested it. Btw please keep in mind that my fork should be considered LGPL as stated in https://github.com/The1only/rosettadrone/pull/76#issuecomment-853044930

A BSD License Agreement for my own changes was only given to The1only