Intel-Media-SDK / MediaSDK

The Intel® Media SDK
MIT License
926 stars 457 forks source link

Calling ConvertMFXParamsToUMC will may change av1's crop size potentailly. #2978

Open zhangyichix opened 2 years ago

zhangyichix commented 2 years ago

The ConvertMFXParamsToUMC is called by VideoDECODEXXXX::init. It will convert user's params to UMC params and use it to init m_decoder. But it makes clip_info same as disp_clip_info.

umcVideoParams->clip_info.height = par->mfx.FrameInfo.Height;
umcVideoParams->clip_info.width = par->mfx.FrameInfo.Width;

umcVideoParams->disp_clip_info.height = umcVideoParams->clip_info.height;  
umcVideoParams->disp_clip_info.width = umcVideoParams->clip_info.width;

Then usually user will call MFXVideoDECODE_GetVideoParam to synchronize params. Different formats of video will call their own GetVideoParam. And GetVideoParam will call their own FillVideoParam as well. Below is av1's. https://github.com/Intel-Media-SDK/MediaSDK/blob/03dd2cc314b291308fb3829f55cceedb8c15efe9/_studio/mfx_lib/decode/av1/src/mfx_av1_dec_decode.cpp#L1101 It will call ConvertUMCParamsToMFX to convert umc params which is saved by init function to mfx params. And let's take a look what ConvertUMCParamsToMFX does.

    par->mfx.FrameInfo.Height = mfx::align2_value(si->clip_info.height, 16);
    par->mfx.FrameInfo.Width  = mfx::align2_value(si->clip_info.width,  16);

    par->mfx.FrameInfo.CropX  = par->mfx.FrameInfo.CropY = 0;
    par->mfx.FrameInfo.CropH  = mfxU16(si->disp_clip_info.height);
    par->mfx.FrameInfo.CropW  = mfxU16(si->disp_clip_info.width);

disp_clip_info will assign to crop directly.

Now I will fully describe what happened. First, user call MFXVideoDECODE_DecodeHeader to parse bitstream header, and return params to user. DecodeHeader will call FillVideoParam, so the mfx's height and width are 16 aligned values. And crop size is actual size. Then, user call init with params which is returned by decodeheader. Msdk call ConvertMFXParamsToUMC makes clip_info same as disp_clip_info with asigned values. At last user call GetVideoParam, he get a 16 aligned crop size.

It only happens on av1(maybe mpeg2 will). Because this value in other formats is calculated. My purpose is that report this issue. So I just committed a simple solution. Actually I don't know how msdk defines clip_info and disp_clip_info.

        ClipInfo            clip_info;                          // (ClipInfo) size of video stream
        ClipInfo            disp_clip_info;