VPP resize generates jumpy frames (for 2:3 pulldown coded video)

[HermanMolinder]

I have a stream which contains mixed PicStructs (combinations of Progressive, TFF, BFF, Field_Repeat). When decoding and resizing through VPP the frames jumps up and down in vertical direction. It also looks like Top and Bottom field gets mixed up.

Reproducable with: sample_multi_transcode -i::mpeg2 video_es.mpeg2 -o::raw out.raw -w 720 -h 480

Input stream: https://a2b-my.sharepoint.com/:u:/g/personal/herman_molinder_wisi_se/Ec0zRVAAyzhLqFj5EcQAZT0B2QcwpbyjVMKM66rXA3t3zA?e=ZaEndf

Without resizing output looks fine. Tested on Media SDK 20.1.0 on both Apollo Lake and Kaby Lake.

[FurongZhang]

We have already worked on that. Will update it soon.

[HermanMolinder]

@FurongZhang , I'm really glad to hear that! When do you estimate this fix will be released?

[VickyZengg]

@HermanMolinder Hi, I checked this issue. You can add deinterlace force VPP to deinterlace input stream. sample_multi_transcode -i::mpeg2 video_es.mpeg2 -o::raw out.raw -w 720 -h 480 -deinterlace::BOB It may help you fix this problem. Thx.

[HermanMolinder]

@VickyZengg , Thank you, but we need to output it as interlaced. It's also a performance concern. So deinterlacing isn't a good solution for our application.

[FurongZhang]

You mean, for the interlaced field, you need to output the interlaced and scaled fields. Right? If yes, for the interlaced fields, we can do some interlaced scaling.

[HermanMolinder]

@FurongZhang Yes, that's right. The output is fed to a PAL/NTSC modulator, which expects an interlaced video that is 720x576/720x480.

[HermanMolinder]

And for this particular stream we check the Field_Repeat flag to decide if some fields are to be repeated to the modulator.

[HermanMolinder]

Hi @FurongZhang , This issue is still present in Media SDK release 20.3.0. You mentioned that you had worked on this issue? Can you please elaborate? BR Herman

[FurongZhang]

Yes, we almost rooted cause this issue. Next Monday, we can give you an update about this issue.

[HermanMolinder]

Thank you. I'm happy to hear that!

[VickyZengg]

Hi @HermanMolinder We rooted cause this issue. In interlaced scale, we take different ways to deal with top field and bottom field. So it looks like jumpy in mixed PicStructs. Top and Bottom field didn't get mixed up. The purpose of this different ways is to make the interlaced scale quality better. Do you have any quality concerns else?

[HermanMolinder]

Hi @VickyZengg , I see, that explains a lot! Do you have a solution for this? I don't have any other quality concerns for now.

[VickyZengg]

Hi @HermanMolinder This behavior is expected, it improves interlaced scale quality. If you don't have other quality concerns, it's no need to change it. If you have, please feel free to let us know.

Thanks, Vicky

[HermanMolinder]

@VickyZengg , I think we misunderstood each other. The input stream (with mixed PicStruct) is a valid MPEG stream and the resulting output has inferior quality which is not expected. I want to claim that this is a bug that needs to be fixed. BR Herman

[VickyZengg]

Hi @HermanMolinder Interlaced content can't be played directly. In interlaced content, we think this jumpy is expected. Is it still jumpy after deinterlaced? Thanks, Vicky

[HermanMolinder]

Hi @VickyZengg , Our application decodes, processes and modulates a PAL/NTSC signal. PAL/NTSC is interlaced and can be viewed on a CRT TV without any deinterlacing involved. This video stream is decoded correctly, without jumpy picture and without deinterlacing, in other professional decoder chipsets. So this is something we really need help solving in Intel Media SDK. Please tell me if you want me to explain more. We can have a video conference meeting if that makes it easier. BR Herman

[VickyZengg]

Hi @HermanMolinder I update one patch : https://github.com/VickyZengg/media-driver Can you use this driver to see if it fix the problem? Waiting for your reply. Thank you. Thanks, Vicky

[HermanMolinder]

Hi @VickyZengg , I tested your patch and still get jumpy picture. BR Herman

[VickyZengg]

Hi @HermanMolinder The jumpy is the same as before?

[HermanMolinder]

@VickyZengg It looked the same, but I need to double check to be sure if it was exactly the same behavior. Need to check that in our office, which I will be back to next week.

[VickyZengg]

Hi @HermanMolinder , it can't be the same. We did test, they are not bitmatch. If you have any problems in compiling driver, please contact me. Thank you.

[HermanMolinder]

Hi @VickyZengg , Sorry for the delay. Haven't been able to test this until now, due to ongoing pandemic..

You're right, with your patch applied the picture looks different. It doesn't look good though. The picture is still jumpy, and for "normal" interlaced content the quality has been reduced. There're combing artefacts. So this actually made it worse.

BR Herman