search

obsproject / obs-studio

OBS Studio - Free and open source software for live streaming and screen recording
https://obsproject.com
GNU General Public License v2.0
60.68k stars 8.02k forks source link

[BUG] I got an error when I set bframes to 0 in Quick Sync Video #3979

Closed shinji3 closed 3 years ago

shinji3 commented 3 years ago

Platform

Operating system and version: Windows 10 20H2 64bit OBS Studio version: 26.1.0 64bit

Expected Behavior

Encoding starts

Current Behavior

Encoding did not start and an error occurred

Steps to Reproduce

  1. Open settings
  2. Use QuickSync H.264
  3. Set B Frames to 0
  4. Record/Stream
  5. Error

    Additional information

    I got an error on this CPU

Celeron N3350 Celeron J3455 Celeron J4115

WizardCM commented 3 years ago

Please post a log file from the session after reproducing the issue (Help -> Log Files -> Upload Current Log)

shinji3 commented 3 years ago

https://obsproject.com/logs/5cC88VPCU6PnRZZE

shinji3 commented 3 years ago

OBS Studio 25.0.8 has no bugs in B Frames 0 for QuickSync H.264 OBS Studio 26.0 has bugs in B Frames 0 for QuickSync H.264

shinji3 commented 3 years ago

QSV_Encoder_Internal.cpp

            if (pParams->nRateControl == MFX_RATECONTROL_CBR ||
                pParams->nRateControl == MFX_RATECONTROL_VBR) {
                m_co2.LookAheadDepth = pParams->nLADEPTH;
            }

This is the cause of the bug

RytoEX commented 3 years ago

I tried to reproduce this on my Intel HD Graphics 630 (i7-7700HQ / Kaby Lake) and I could not get it to fail with B Frames set to 0. Maybe this is a peculiarity of the J4115, or more broadly with Apollo Lake and Gemini Lake chips (Goldmont / Goldmont Plus)?

For reference, I believe this is the code block referred to in the above comment, which was indeed changed for OBS Studio 26.0.0 in commit https://github.com/obsproject/obs-studio/commit/28af4533206d4112c4751b670563d590f8c480a4 by @brittneysclark: https://github.com/obsproject/obs-studio/blob/28af4533206d4112c4751b670563d590f8c480a4/plugins/obs-qsv11/QSV_Encoder_Internal.cpp#L282-L288

Does it still fail if you select another Rate Control Method such as CQP? How about with LA_ICQ?

shinji3 commented 3 years ago

Latency normal and B Frames 0 CBR = failed VBR = failed VCM = succeeded CQP = succeeded AVBR = succeeded ICQ = succeeded LA_ICQ = failed LA_CBR = failed LA_VBR = failed

Latency ultra-low and B Frames 0 CBR = succeeded VBR = succeeded VCM = succeeded CQP = succeeded AVBR = succeeded ICQ = succeeded LA_ICQ = succeeded LA_CBR = succeeded LA_VBR = succeeded

RytoEX commented 3 years ago

Perhaps LookAhead and B Frames aren't playing nice on certain platforms. See also this code block above the one previously linked: https://github.com/obsproject/obs-studio/blob/28af4533206d4112c4751b670563d590f8c480a4/plugins/obs-qsv11/QSV_Encoder_Internal.cpp#L275-L277

Using "ultra-low" latency sets nLADEPTH to 0. Using "low" latency sets nLADEPTH to FPS / 2. Using "normal" latency sets nLADEPTH to FPS. This value is later processed further.

In your case, with 30 FPS, nLADEPTH is set to 30 and left at 30. For ultra-low latency, nLADEPTH would become 10. For low latency, it would become 15.

https://github.com/obsproject/obs-studio/blob/d04e167ed09cf9140dcc27d11cc792cbf9c97f08/plugins/obs-qsv11/obs-qsv11.c#L486-L503

I'm still not clear on why nLADEPTH 30 would cause this to fail on the platforms you reported, so someone else will have to shed light on this.

shinji3 commented 3 years ago

Fixed a bug where

bool QSV_Encoder_Internal::InitParams(qsv_param_t *pParams)
{
    memset(&m_mfxEncParams, 0, sizeof(m_mfxEncParams));

    m_mfxEncParams.mfx.CodecId = MFX_CODEC_AVC;
    m_mfxEncParams.mfx.GopOptFlag = MFX_GOP_STRICT;
    m_mfxEncParams.mfx.NumSlice = 1;
    m_mfxEncParams.mfx.TargetUsage = pParams->nTargetUsage;
    m_mfxEncParams.mfx.CodecProfile = pParams->nCodecProfile;
    m_mfxEncParams.mfx.FrameInfo.FrameRateExtN = pParams->nFpsNum;
    m_mfxEncParams.mfx.FrameInfo.FrameRateExtD = pParams->nFpsDen;
    m_mfxEncParams.mfx.FrameInfo.FourCC = MFX_FOURCC_NV12;
    m_mfxEncParams.mfx.FrameInfo.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
    m_mfxEncParams.mfx.FrameInfo.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
    m_mfxEncParams.mfx.FrameInfo.CropX = 0;
    m_mfxEncParams.mfx.FrameInfo.CropY = 0;
    m_mfxEncParams.mfx.FrameInfo.CropW = pParams->nWidth;
    m_mfxEncParams.mfx.FrameInfo.CropH = pParams->nHeight;
    m_mfxEncParams.mfx.GopRefDist = pParams->nbFrames + 1;

    enum qsv_cpu_platform qsv_platform = qsv_get_cpu_platform();
    if ((qsv_platform >= QSV_CPU_PLATFORM_ICL) &&
        (pParams->nbFrames == 0) &&
        (m_ver.Major == 1 && m_ver.Minor >= 31) &&
        (pParams->nRateControl != MFX_RATECONTROL_LA_ICQ &&
         pParams->nRateControl != MFX_RATECONTROL_LA)) {
        m_mfxEncParams.mfx.LowPower = MFX_CODINGOPTION_ON;
    }

    m_mfxEncParams.mfx.RateControlMethod = pParams->nRateControl;

    switch (pParams->nRateControl) {
    case MFX_RATECONTROL_CBR:
        m_mfxEncParams.mfx.TargetKbps = pParams->nTargetBitRate;
        break;
    case MFX_RATECONTROL_VBR:
    case MFX_RATECONTROL_VCM:
        m_mfxEncParams.mfx.TargetKbps = pParams->nTargetBitRate;
        m_mfxEncParams.mfx.MaxKbps = pParams->nMaxBitRate;
        break;
    case MFX_RATECONTROL_CQP:
        m_mfxEncParams.mfx.QPI = pParams->nQPI;
        m_mfxEncParams.mfx.QPB = pParams->nQPB;
        m_mfxEncParams.mfx.QPP = pParams->nQPP;
        break;
    case MFX_RATECONTROL_AVBR:
        m_mfxEncParams.mfx.TargetKbps = pParams->nTargetBitRate;
        m_mfxEncParams.mfx.Accuracy = pParams->nAccuracy;
        m_mfxEncParams.mfx.Convergence = pParams->nConvergence;
        break;
    case MFX_RATECONTROL_ICQ:
        m_mfxEncParams.mfx.ICQQuality = pParams->nICQQuality;
        break;
    case MFX_RATECONTROL_LA:
        m_mfxEncParams.mfx.TargetKbps = pParams->nTargetBitRate;
        break;
    case MFX_RATECONTROL_LA_ICQ:
        m_mfxEncParams.mfx.ICQQuality = pParams->nICQQuality;
        break;
    case MFX_RATECONTROL_LA_HRD:
        m_mfxEncParams.mfx.TargetKbps = pParams->nTargetBitRate;
        m_mfxEncParams.mfx.MaxKbps = pParams->nTargetBitRate;
        break;
    default:
        break;
    }

    m_mfxEncParams.AsyncDepth = pParams->nAsyncDepth;
    m_mfxEncParams.mfx.GopPicSize =
        (mfxU16)(pParams->nKeyIntSec * pParams->nFpsNum /
             (float)pParams->nFpsDen);

    static mfxExtBuffer *extendedBuffers[3];
    int iBuffers = 0;

    if (m_ver.Major == 1 && m_ver.Minor >= 8) {
        memset(&m_co2, 0, sizeof(mfxExtCodingOption2));
        m_co2.Header.BufferId = MFX_EXTBUFF_CODING_OPTION2;
        m_co2.Header.BufferSz = sizeof(m_co2);
        if (pParams->nRateControl == MFX_RATECONTROL_LA_ICQ ||
            pParams->nRateControl == MFX_RATECONTROL_LA)
            m_co2.LookAheadDepth = pParams->nLADEPTH;
        if (pParams->bMBBRC)
            m_co2.MBBRC = MFX_CODINGOPTION_ON;
        if (pParams->nbFrames > 1)
            m_co2.BRefType = MFX_B_REF_PYRAMID;
        if (m_mfxEncParams.mfx.LowPower == MFX_CODINGOPTION_ON) {
            m_co2.RepeatPPS = MFX_CODINGOPTION_OFF;
        }
        extendedBuffers[iBuffers++] = (mfxExtBuffer *)&m_co2;
    }

    if (m_mfxEncParams.mfx.LowPower == MFX_CODINGOPTION_ON) {
        memset(&m_co3, 0, sizeof(mfxExtCodingOption3));
        m_co3.Header.BufferId = MFX_EXTBUFF_CODING_OPTION3;
        m_co3.Header.BufferSz = sizeof(m_co3);
        m_co3.ScenarioInfo = MFX_SCENARIO_GAME_STREAMING;
        extendedBuffers[iBuffers++] = (mfxExtBuffer *)&m_co3;
    } else if (pParams->bCQM) {
        if (m_ver.Major == 1 && m_ver.Minor >= 16) {
            memset(&m_co3, 0, sizeof(mfxExtCodingOption3));
            m_co3.Header.BufferId = MFX_EXTBUFF_CODING_OPTION3;
            m_co3.Header.BufferSz = sizeof(m_co3);
            m_co3.ScenarioInfo = 7; // MFX_SCENARIO_GAME_STREAMING
            extendedBuffers[iBuffers++] = (mfxExtBuffer *)&m_co3;
        }
    }

    if (iBuffers > 0) {
        m_mfxEncParams.ExtParam = extendedBuffers;
        m_mfxEncParams.NumExtParam = (mfxU16)iBuffers;
    }

    // Width must be a multiple of 16
    // Height must be a multiple of 16 in case of frame picture and a
    // multiple of 32 in case of field picture
    m_mfxEncParams.mfx.FrameInfo.Width = MSDK_ALIGN16(pParams->nWidth);
    m_mfxEncParams.mfx.FrameInfo.Height = MSDK_ALIGN16(pParams->nHeight);

    if (m_bUseD3D11 || m_bD3D9HACK)
        m_mfxEncParams.IOPattern = MFX_IOPATTERN_IN_VIDEO_MEMORY;
    else
        m_mfxEncParams.IOPattern = MFX_IOPATTERN_IN_SYSTEM_MEMORY;

    mfxStatus sts = m_pmfxENC->Query(&m_mfxEncParams, &m_mfxEncParams);
    if (sts == MFX_ERR_UNSUPPORTED || sts == MFX_ERR_UNDEFINED_BEHAVIOR) {
        m_mfxEncParams.mfx.LowPower = MFX_CODINGOPTION_OFF;
    }

    return true;
}
cmlin2 commented 3 years ago

I have added a patch to fix the issue uncovered by @shinji3. Please let me know if it resolves the issues on your systems.

shinji3 commented 3 years ago

It worked in my system Thank you for fixing the bug

cmlin2 commented 3 years ago

Wonderful, thanks @shinji3!