[ffmpeg filter zscale][avx-512] color looks pale when using >2-taps filter combined with matrix or primaries conversion

[Mr-Z-2697]

I use media-autobuild_suite to build my ffmpeg, so it should be up-to-date with git. Last time I can confirm this problem doesn't appear is 2022.09. This also comes to ffplay, which I used as a quick way to demostrate this problem. First image was with -vf zscale=p=bt470bg:f=spline16, second one was with -vf zscale=p=bt470bg:f=spline36. Based on my tests, only spline and lanczos with >2-taps have this issue, and transfer conversion is safe while matrix and primaries are not, but I only tested bt709 sources. And sorry I'm bad at organizing my words. It may be a little hard to read.

[Mr-Z-2697]

Wait wait wait... I ran into this issue with ryzen 7950x, now I tested it on my i5-7400 and seems no problem. (and back when 2022.09 I was using ryzen 5950x) So maybe the last commit about zen4 affected this?

[Mr-Z-2697]

I'll try build to previous commit.

[Mr-Z-2697]

Built to commit fc993b40effd6536518444fcbd435b825b30ca73, no change. Or is it avx-512 related?

[Mr-Z-2697]

So I tried to "disable" avx-512 and build, results are fine now. I don't know how to disable avx-512 through media-autobuild_suite or even manually configure, so I tried to delete these code from configure.ac and it seemed working. https://github.com/sekrit-twc/zimg/blob/48eec5c68b086bbae0f75498e25b5a229e5ea684/configure.ac#L91-L108 I let media-autobuild_suite check git update for zimg, and swapped zimg files in "lib" and "include" folder.

[Mr-Z-2697]

I tested with vultr intel avx-512 instances and same thing happens. Looks like there's something going on in the avx-512 optimization code.

[sekrit-twc]

Does this happen in the v3.0 branch or only in master?

[sekrit-twc]

Also, can you confirm which operations are required to reproduce the issue?

• 4:2:0 input format? or RGB?
• 4:2:0 output format? or RGB?
• >2-tap resample filter
• color matrix change
• transfer characteristics change
• primaries change

If you can use VapourSynth + BlankClip to create a repro, that would be appreciated.

[Mr-Z-2697]

Also, can you confirm which operations are required to reproduce the issue?

• 4:2:0 input format? or RGB?
• 4:2:0 output format? or RGB?
• >2-tap resample filter
• color matrix change
• transfer characteristics change
• primaries change

If you can use VapourSynth + BlankClip to create a repro, that would be appreciated.

• Input format could be any, but output should include horizontal subsampling (as yuv422 has problem and yuv440 don't)
• >2-tap resample filter is required as shown in first post (showing spline16 and spline36)
• Color matrix or color primaries change is required
• Transfer characteristics change is not required

I use VapourSynth R61 and it doesn't have this issue. As for git version, I didn't tried build VapourSynth on Windows, sorry.

[Mr-Z-2697]

Does this happen in the v3.0 branch or only in master?

v3.0 branch don't have this issue

[sekrit-twc]

Was able to reproduce in VS:

c = core.std.BlankClip(format=vs.RGB48, color=[0x8000, 0x4000, 0xC000])
c = core.resize.Lanczos(c, format=vs.YUV420P10, matrix_s='709', filter_param_a=3)
c.set_output()

With 3+ taps, UV plane output goes from uniform [710, 603] to non-uniform with average of [600, 553], which means there is some kind of memory corruption.

EDIT: Even more minimal repro

c = core.std.BlankClip(format=vs.RGBS, color=[0x8000/0xFFFF, 0x4000/0xFFFF, 0xC000/0xFFFF])
c = core.resize.Lanczos(c, format=core.query_video_format(vs.YUV, vs.FLOAT, 32, 1, 0).id, matrix_s='709', filter_param_a=3)