ROI feature not work well on QSV

lion117 commented 4 years ago

computer

intel I5-6500
intel media sdk 2018-R2
use hw hevc and hw h264

STATE

encoder	bitrate mode	state
h264	CQP	work well
h264	CBR	not support (-3)
hevc	CQP	support , but roi not work
hevc	CBR	support , but roi not work

dmitryermilov commented 4 years ago

@saosipov , @ilapichev , please respond

saosipov commented 4 years ago

Hi lion117 As I remember SKL supports ROI for BRC only and user can set Priority for these regions. Could you please share here your command line along with output log? -Sergey

lion117 commented 4 years ago

I used official demo sample_encode and modify the code to test the roi featrure . Here is My Code

 memset(&m_cfgROISetting, 0, sizeof(m_cfgROISetting));
            m_cfgROISetting.Header.BufferId = MFX_EXTBUFF_ENCODER_ROI;;
            m_cfgROISetting.Header.BufferSz = sizeof(m_cfgROISetting);

            m_cfgROISetting.NumROI = 1;
            m_cfgROISetting.ROIMode = MFX_ROI_MODE_PRIORITY;
            m_cfgROISetting.ROI[0].Left = 160;
            m_cfgROISetting.ROI[0].Right = 640;
            m_cfgROISetting.ROI[0].Top = 0;
            m_cfgROISetting.ROI[0].Bottom = 320;
            m_cfgROISetting.ROI[0].DeltaQP = -51;
            //m_cfgROISetting.ROI[0].Priority = 3;
            m_EncExtParams.push_back((mfxExtBuffer *)&m_cfgROISetting);

Here Is the Roi working video shootscreen only on H264CPQ ratecontrol

img1

vlcsnap-2020-02-08-15h47m30s130

企业微信截图_15789037125652

img2

lion117 commented 4 years ago

Hi lion117 As I remember SKL supports ROI for BRC only and user can set Priority for these regions. Could you please share here your command line along with output log? -Sergey

Hi, Do you have any progress?

lakulako commented 4 years ago

Hi lion117, Thank you for using msdk. There is a conflict in your code: you set MFX_ROI_MODE_PRIORITY together with DeltaQP value -51. Values for priority are limited to [-3,3] and so initialization fails with INVALID_PARAM. Note, that DeltaQP and Priority fields make union. Please set MFX_ROI_MODE_QP_DELTA instead. BTW, "Priority is deprecated mode and is used only for backward compatibility". All the best, -lakulako

lion117 commented 4 years ago

Hi lion117, Thank you for using msdk. There is a conflict in your code: you set MFX_ROI_MODE_PRIORITY together with DeltaQP value -51. Values for priority are limited to [-3,3] and so initialization fails with INVALID_PARAM. Note, that DeltaQP and Priority fields make union. Please set MFX_ROI_MODE_QP_DELTA instead. BTW, "Priority is deprecated mode and is used only for backward compatibility". All the best, -lakulako

Hi, I had tried two different cases which both return invalid para when I used qsv264 cbr bitrate control mode .

case 1

memset(&m_cfgROISetting, 0, sizeof(m_cfgROISetting));
            m_cfgROISetting.Header.BufferId = MFX_EXTBUFF_ENCODER_ROI;;
            m_cfgROISetting.Header.BufferSz = sizeof(m_cfgROISetting);

            m_cfgROISetting.NumROI = 1;
            m_cfgROISetting.ROIMode = MFX_ROI_MODE_PRIORITY;
            m_cfgROISetting.ROI[0].Left = 160;
            m_cfgROISetting.ROI[0].Right = 640;
            m_cfgROISetting.ROI[0].Top = 0;
            m_cfgROISetting.ROI[0].Bottom = 320;
            m_cfgROISetting.ROI[0].Priority = 3;
            m_EncExtParams.push_back((mfxExtBuffer *)&m_cfgROISetting);

case 2

memset(&m_cfgROISetting, 0, sizeof(m_cfgROISetting));
            m_cfgROISetting.Header.BufferId = MFX_EXTBUFF_ENCODER_ROI;;
            m_cfgROISetting.Header.BufferSz = sizeof(m_cfgROISetting);

            m_cfgROISetting.NumROI = 1;
            m_cfgROISetting.ROIMode = MFX_ROI_MODE_QP_DELTA ;
            m_cfgROISetting.ROI[0].Left = 160;
            m_cfgROISetting.ROI[0].Right = 640;
            m_cfgROISetting.ROI[0].Top = 0;
            m_cfgROISetting.ROI[0].Bottom = 320;
            m_cfgROISetting.ROI[0].DeltaQP = -51;
            m_EncExtParams.push_back((mfxExtBuffer *)&m_cfgROISetting);

lakulako commented 4 years ago

Hi, hard to understand what is going on in your case without information. Could you please provide command line, logs and binaries' version?

lion117 commented 4 years ago

Hi, hard to understand what is going on in your case without information. Could you please provide command line, logs and binaries' version?

intel I5-6500
intel media sdk 2018-R2 and 2019-R1
use hw hevc and hw h264

I used the Encode_sample code to test ROI feature. Test video Roi by elecard streameye demo .

lakulako commented 4 years ago

And what is the command line? Logs? (for sample_encode you can use -dump to dump configuration after init.

lion117 commented 4 years ago

And what is the command line? Logs? (for sample_encode you can use -dump to dump configuration after init.

I modified Sample_encode code to make roi feature work , instead of by cmdline . So roi region was fixed. there were no Logs or dumps

# h265
sample_encode.exe h265 -i  singer.yuv -o singer_out_265_1200.h265 -w 960 -h 720  -cbr -b 1200  -f 15  -g  30

# h264
sample_encode.exe h264 -i  singer.yuv -o singer_out_264_1200.h264 -w 960 -h 720  -cbr -b 1200  -f 15  -g  30

My question is

Why qsv264 cbr bitrate mode ROI not support when I init sdk
qsv265 roi did not work at all . Couldn't U Reproduce those issues?

lakulako commented 4 years ago

Let's try with sample_encode. It is more informative. Lets continue with it.

Does sample_encode prints anything to screen? It is log that I asked. Please try adding "-dump fname" and share fname.

lakulako commented 4 years ago

@lion117 , have you resolved the issue?

lion117 commented 4 years ago

No

------------------ 原始邮件 ------------------ 发件人: "Leonid Kulakov"<notifications@github.com>; 发送时间: 2020年2月27日(星期四) 晚上8:24 收件人: "Intel-Media-SDK/MediaSDK"<MediaSDK@noreply.github.com>; 抄送: "leo_nardo"<ngixpro@qq.com>; "Mention"<mention@noreply.github.com>; 主题: Re: [Intel-Media-SDK/MediaSDK] ROI feature not work well on QSV (#1904)

@lion117 , have you resolved the issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

lion117 commented 4 years ago

Let's try with sample_encode. It is more informative. Lets continue with it.

Does sample_encode prints anything to screen? It is log that I asked. Please try adding "-dump fname" and share fname.

Couldn't U Reproduce the ROI Issue? was everything working very well when U tested ?

lakulako commented 4 years ago

ROI works well on my side, but with different application. Sample encode doesn't support ROI. I will not try sample_encode application with your modifications. I could look into logs that you don't share.

lion117 commented 4 years ago

I would like to provide my roi verion sample_encode source code for you . would It be helpful for u to analysis the issue ?

lakulako commented 4 years ago

Yes, this is what I asked. Better provide this as patch to msdk sample_encode.

lion117 commented 4 years ago

hi, I uploaded the source code into my cloud bucket.

src code

yuv data

roi core code . which u you search in src code

        memset(&m_cfgROISetting, 0, sizeof(m_cfgROISetting));
         m_cfgROISetting.Header.BufferId = MFX_EXTBUFF_ENCODER_ROI;;
        m_cfgROISetting.Header.BufferSz = sizeof(m_cfgROISetting);

        m_cfgROISetting.NumROI = 1;
        m_cfgROISetting.ROIMode = MFX_ROI_MODE_PRIORITY;
        m_cfgROISetting.ROI[0].Left = 160;
        m_cfgROISetting.ROI[0].Right = 640;
        m_cfgROISetting.ROI[0].Top = 160;
        m_cfgROISetting.ROI[0].Bottom = 320;
        m_cfgROISetting.ROI[0].DeltaQP = -51;

Any question about the source code , please let me know

lion117 commented 4 years ago

hi , are there any I can help ?

mpushki commented 4 years ago

Hi @lion117 , I couldn't build your solution, so I will add your changes to our sample_encode and investigate your issue.

lion117 commented 4 years ago

Hi @lion117 , I couldn't build your solution, so I will add your changes to our sample_encode and investigate your issue.

HI, I rebuilded RunIntelDemo.sln which builded successfully. Could you show the error code ? Maybe I can do some help. had your visual studio 2013 installed update 5?

mpushki commented 4 years ago

I use VS 2017. i5-6300(SKL) I added your code:

        std::memset(&m_cfgROISetting, 0, sizeof(m_cfgROISetting));
        m_cfgROISetting.Header.BufferId = MFX_EXTBUFF_ENCODER_ROI;
        m_cfgROISetting.Header.BufferSz = sizeof(m_cfgROISetting);

        m_cfgROISetting.NumROI = 1;
        m_cfgROISetting.ROIMode = MFX_ROI_MODE_QP_DELTA;
        m_cfgROISetting.ROI[0].Left = 160;
        m_cfgROISetting.ROI[0].Right = 640;
        m_cfgROISetting.ROI[0].Top = 160;
        m_cfgROISetting.ROI[0].Bottom = 320;
        m_cfgROISetting.ROI[0].DeltaQP = -51;

        m_EncExtParams.push_back((mfxExtBuffer *)&m_cfgROISetting);

in CEncodingPipeline::InitMfxEncParams sample_decode from MediaSDK.

AVC it works fine for -cqp and -cbr:

sample_encode.exe h264 -i 2645142_960x720.yuv -o test.h264 -w 960 -h 720  -cqp -qpi 30 -qpb 32 -qpp 32 -f 15 -g 30
sample_encode.exe h264 -i 2645142_960x720.yuv -o test.h264 -w 960 -h 720  -cbr -b 1200 -f 15 -g 30

HEVC Windows SKL doesn't support ROI for HEVC.

I run this cmd on ICL - and it works.

sample_encode.exe h265 -i 2645142_960x720.yuv -o test.h265 -w 960 -h 720  -cqp -qpi 30 -qpb 32 -qpp 32 -f 15 -g 30
sample_encode.exe h265 -i 2645142_960x720.yuv -o test.h265 -w 960 -h 720  -cbr -b 1200 -f 15 -g 30

For additional information: Linux SKL supports ROI for both of codecs.

I hope it helps you.

lion117 commented 4 years ago

Thanks , I will take a try for the AVC operation. By the way , I have a question about how do you run the cmd on ICL.

mpushki commented 4 years ago

@lion117, ICL is next generation of CPU , for example I tested on i7-1060G7. cmd was the same.

lion117 commented 4 years ago

I use VS 2017. i5-6300(SKL) I added your code:

        std::memset(&m_cfgROISetting, 0, sizeof(m_cfgROISetting));
        m_cfgROISetting.Header.BufferId = MFX_EXTBUFF_ENCODER_ROI;
        m_cfgROISetting.Header.BufferSz = sizeof(m_cfgROISetting);

        m_cfgROISetting.NumROI = 1;
        m_cfgROISetting.ROIMode = MFX_ROI_MODE_QP_DELTA;
        m_cfgROISetting.ROI[0].Left = 160;
        m_cfgROISetting.ROI[0].Right = 640;
        m_cfgROISetting.ROI[0].Top = 160;
        m_cfgROISetting.ROI[0].Bottom = 320;
        m_cfgROISetting.ROI[0].DeltaQP = -51;

        m_EncExtParams.push_back((mfxExtBuffer *)&m_cfgROISetting);

in CEncodingPipeline::InitMfxEncParams sample_decode from MediaSDK.

AVC it works fine for -cqp and -cbr:

sample_encode.exe h264 -i 2645142_960x720.yuv -o test.h264 -w 960 -h 720  -cqp -qpi 30 -qpb 32 -qpp 32 -f 15 -g 30
sample_encode.exe h264 -i 2645142_960x720.yuv -o test.h264 -w 960 -h 720  -cbr -b 1200 -f 15 -g 30

HEVC Windows SKL doesn't support ROI for HEVC.

I run this cmd on ICL - and it works.

sample_encode.exe h265 -i 2645142_960x720.yuv -o test.h265 -w 960 -h 720  -cqp -qpi 30 -qpb 32 -qpp 32 -f 15 -g 30
sample_encode.exe h265 -i 2645142_960x720.yuv -o test.h265 -w 960 -h 720  -cbr -b 1200 -f 15 -g 30

For additional information: Linux SKL supports ROI for both of codecs.

I hope it helps you.

HI, I had tried the method which you mentioned. But It still didnot work. I Add the Roi code in the sample_encode demo on the CEncodingPipeline::InitMfxEncParams function.

Could you provide the sample code And the sample_encode.exe for me ?

小Q截图-20200410153558

lion117 commented 4 years ago

@lion117, ICL is next generation of CPU , for example I tested on i7-1060G7. cmd was the same.

Hi, Are you still on call ?

mpushki commented 4 years ago

Hi @lion117 Sorry for delay, I will prepare sample code which should works today

lion117 commented 4 years ago

Hi @lion117 Sorry for delay, I will prepare sample code which should works today

please send the sample code to my email ngixpro@qq.com

mpushki commented 4 years ago

@lion117 I noticed that source code was changed and my previous code needs to correct. I need more time to understand what was changed and test ROI again.

mpushki commented 4 years ago

@lion117 Could please download last version of MediaSDK. And add next ROI code to pipeline_encode.cpp:

        auto roi = m_mfxEncParams.AddExtBuffer<mfxExtEncoderROI>();
        roi->Header.BufferId = MFX_EXTBUFF_ENCODER_ROI;
        roi->Header.BufferSz = sizeof(roi);

        roi->NumROI = 1;
        roi->ROIMode = MFX_ROI_MODE_QP_DELTA;
        roi->ROI[0].Left = 160;
        roi->ROI[0].Right = 640;
        roi->ROI[0].Top = 160;
        roi->ROI[0].Bottom = 320;
        roi->ROI[0].DeltaQP = -51;

Also add changes to sample_utils.h to struct mfx_ext_buffer_id:

template<>struct mfx_ext_buffer_id<mfxExtEncoderROI>{
    enum {id = MFX_EXTBUFF_ENCODER_ROI};
};

It works for AVC -cbr

mpushki commented 4 years ago

@lion117 Could please download last version of MediaSDK. And add next ROI code to pipeline_encode.cpp:

        auto roi = m_mfxEncParams.AddExtBuffer<mfxExtEncoderROI>();
        roi->Header.BufferId = MFX_EXTBUFF_ENCODER_ROI;
        roi->Header.BufferSz = sizeof(roi);

        roi->NumROI = 1;
        roi->ROIMode = MFX_ROI_MODE_QP_DELTA;
        roi->ROI[0].Left = 160;
        roi->ROI[0].Right = 640;
        roi->ROI[0].Top = 160;
        roi->ROI[0].Bottom = 320;
        roi->ROI[0].DeltaQP = -51;

Also add changes to sample_utils.h to struct mfx_ext_buffer_id:

template<>struct mfx_ext_buffer_id<mfxExtEncoderROI>{
    enum {id = MFX_EXTBUFF_ENCODER_ROI};
};

It works for AVC -cbr

lion117 commented 4 years ago

@lion117 Could please download last version of MediaSDK. And add next ROI code to pipeline_encode.cpp:

        auto roi = m_mfxEncParams.AddExtBuffer<mfxExtEncoderROI>();
        roi->Header.BufferId = MFX_EXTBUFF_ENCODER_ROI;
        roi->Header.BufferSz = sizeof(roi);

        roi->NumROI = 1;
        roi->ROIMode = MFX_ROI_MODE_QP_DELTA;
        roi->ROI[0].Left = 160;
        roi->ROI[0].Right = 640;
        roi->ROI[0].Top = 160;
        roi->ROI[0].Bottom = 320;
        roi->ROI[0].DeltaQP = -51;

Also add changes to sample_utils.h to struct mfx_ext_buffer_id:

template<>struct mfx_ext_buffer_id<mfxExtEncoderROI>{
    enum {id = MFX_EXTBUFF_ENCODER_ROI};
};

It works for AVC -cbr

I had tried IntelMediaSdk 2018 R2 and IntelMediaSdk 2019R1 . This solution was still not work that hinted -3 not supoort Could u provide the sample code or exe which could work?
I think that would be more efficient

mpushki commented 4 years ago

Please try this exe file sample_encode.zip

lion117 commented 4 years ago

Please try this exe file sample_encode.zip

I had tested the sample_encode.exe on windows 7 that you provided which still not worked.

Are there any solution that could make the qsv roi work on windows without relaying on any specified environment ?

mpushki commented 4 years ago

Do you have the latest video driver installed on your machine?

lion117 commented 4 years ago

what kind of video driver ?

mpushki commented 4 years ago

This is mine for exapmle.

lion117 commented 4 years ago

yes , I had installed the latest graphic driver

mpushki commented 4 years ago

I've checked intel.com and found more recent version -> https://downloadcenter.intel.com/download/29475/Intel-Graphics-Driver-for-Windows-15-45-?wapkw=intel%20hd%20graphics

Please try to install and report results with new driver.

lion117 commented 4 years ago

I think graphic card driver is not an issue. Even I updgraded it to the latest version . It was still not working . Maybe you should test your code on a new computer.

mpushki commented 4 years ago

I've tested code on new computer and it works too.

mpushki commented 4 years ago

@lion117 Could you please install Window 10, 64 bits? And try to run sample_encode on new OS. Also Microsoft's support for Windows 7 ended on January 14, 2020, I suggest to move to Win 10 anyway.

lion117 commented 4 years ago

@lion117 Could you please install Window 10, 64 bits? And try to run sample_encode on new OS. Also Microsoft's support for Windows 7 ended on January 14, 2020, I suggest to move to Win 10 anyway.

I had tried the sample_encode.exe on win10 which cpu was intel I7-9700. The result was still -3 (not supported).

lion117 commented 4 years ago

I've tested code on new computer and it works too. Had the roi feature effected on the video ? Are there any solution to make the sample_ecode.exe work on more computers?

mpushki commented 4 years ago

Hi @lion117, Could you please rebuild libmfx_vs2015.lib with lasted changes from MediaSDK repo and send me your caps

mpushki commented 4 years ago

@lion117 Could you please install Window 10, 64 bits? And try to run sample_encode on new OS. Also Microsoft's support for Windows 7 ended on January 14, 2020, I suggest to move to Win 10 anyway.

I had tried the sample_encode.exe on win10 which cpu was intel I7-9700. The result was still -3 (not supported).

Have you updated driver on new machine?

mpushki commented 4 years ago

Also could you please put breakpoint to Query, and find line (from lib) which return error. Maybe problem is not in ROI.

lion117 commented 4 years ago

Could you please rebuild libmfx_vs2015.lib with lasted changes from MediaSDK repo and send me your caps

I don't think the unsupported error has relationship with libmfx_vs2015.lib , because I used the sample_encode.exe which had link the latest libmfx.lib .

Have you updated driver on new machine?

yes

lion117 commented 4 years ago

Also could you please put breakpoint to Query, and find line (from lib) which return error. Maybe problem is not in ROI.

I had tried to prohibit the roi code . then it worked. if roi was enabled , when call query api, it return -3 error

mpushki commented 4 years ago

Please check Query parameters and compare src, dst

Intel-Media-SDK / MediaSDK