Request support for YUV444P10/P16 format

[BlackMickey]

Thank you for your great work. I have some videos in YUV444P10 format, so I hope the filter will add support for YUV444P10/P16 format.

I have read this discussion #34 and know that AviSynth+ does not support Y410, Y416 formats yet. Is it possible for VapourSynth Filter to support Y410 and Y416 formats?

[CrendKing]

Both AviSynth+ and VapourSynth only support YUV444P 8/10/16 formats. These are different to Y410 and Y416 isn't it in that the former have 3 planes, while the latter have 4 planes (YUV + alpha). Even if I write conversion logic to extract the YUV planes from Y416, I will have to discard the alpha, making it essentially YUV444P8. Unless there is some sort of alpha blending to upscale the YUV planes to higher bits, which I'm not aware of at hand.

Correct me if I'm wrong here?

[CrendKing]

I noticed that AviSynth+ actually supports YUVA444 formats. YUVA444P16 is basically the planar form of Y416, so I added support for Y416 at https://github.com/CrendKing/avisynth_filter/actions/runs/2178550598. Please test.

Y410 on the other hand is different. It only has 2 bits for the alpha, while YUVA444P10 needs 10 bits. I don't think it's worth doing the pixel-wise upscaling the alpha bits.

VapourSynth doesn't support YUVA.

Make sure you use LAV Filters and madVR to use Y416.

[BlackMickey]

I tried to ask for support on the alpha plane in the VapourSynth project in Github. vapoursynth/vapoursynth#859 The response was "Vapoursynth already supports alpha, but extra plane handled separately." I read the document and it seems that the alpha plane is stored in the reserved frame attribute. frame _Alpha

Thanks for your work, I will test the AviSynth filter with Y416 support in a few days.

[CrendKing]

Interesting. The alpha plane is a completely separate frame huh? I want to say I'll see what I can do, but I realized that even if I am able to implement that, there is no way to verify. Do you have video sample that has non-full alpha plane?

I tried to set the whole alpha to 0, and the video played as if the alpha are all 0xffff. So either my whole understanding of the format is wrong, or my player or video file does not support it.

[BlackMickey]

If the input format of the filter is P216, the SVP will not show an error, but when the format is Y416, the SVP will show an error. Avisynth+ 3.7.1 test build 22 (20211022)

1.

AvsFilterSource()     # YUVA444P16 (Y416)
ConvertToYUV420()     # YUV420P16
input_m = last
input_m8 = ConvertBits(8) 
input_m8.Subtitle(
    \   "\nPixelType = " + String(PixelType(input_m8))
    \ + "\nIsPlanar = " + String(IsPlanar(input_m8))
    \ + "\nHasAlpha = " + String(HasAlpha(input_m8))
    \ + "\nBitsPerComponent = " + String(BitsPerComponent(input_m8))
    \ , font="courier", text_color=$ffffff, size=32, align=9, lsp=0)     # YUV420P8 (YV12)

super     = SVSuper(input_m8, super_params)    # SVSuper: Clip must be YV12

2.

AvsFilterSource()     # YUVA444P16 (Y416)
ConvertToYUV420()     # YUV420P16
input_m = last
input_m8 = ConvertBits(8) 
input_m8.Subtitle(
    \   "\nPixelType = " + String(PixelType(input_m8))
    \ + "\nIsPlanar = " + String(IsPlanar(input_m8))
    \ + "\nHasAlpha = " + String(HasAlpha(input_m8))
    \ + "\nBitsPerComponent = " + String(BitsPerComponent(input_m8))
    \ , font="courier", text_color=$ffffff, size=32, align=9, lsp=0)     # YUV420P8 (YV12) ?

input_m8 = input_m8.ConvertToYV12()     # YUV420P8 (YV12)
super    = SVSuper(input_m8, super_params) 
vectors  = SVAnalyse(super, analyse_params, src=input_m8)
smooth   = SVSmoothFps(input_m, super, vectors, smoothfps_params, mt=threads, src=input_m)    # SVSmoothFps: source must be YUV 4:2:0 8/10/16-bits

[CrendKing]

Converting YUVA444P16 with ConvertToYUV420() results YUVA420P16 (notice the alpha plane). I'm not aware of any "non-destructive" (meaning no reduction of subsampling density or bits) way to simply remove that alpha plane to keep YUV420P16.

To make SVP work, you have to convert it to YV12 (with ConvertBits followed by ConvertToYV12) for their SVSuper and SVSmoothFps functions. Obviously, this ends up with just YUV420P8 video.

On the other hand, I also implemented YUVA444P16 in VPSF. Since in VapourSynth the alpha plane is never part of the main frame itself but only attached, it is pretty much the same as YUV444P16. Converting it to YUV420P16 works, so you can play it with SVP.

Since I'm still unable to see the effect of the alpha plane, I might as well ignore the plane. This way both AVSF and VPSF's 4:4:4 are YUV444Px, which is also SVP-friendly.

[CrendKing]

Try https://github.com/CrendKing/avisynth_filter/actions/runs/2187059799. Should be working fine with SVP (result in P016 as expected). Both AVSF and VPSF supported.

[BlackMickey]

Converting YUVA444P16 with ConvertToYUV420() results YUVA420P16 (notice the alpha plane). I'm not aware of any "non-destructive" (meaning no reduction of subsampling density or bits) way to simply remove that alpha plane to keep YUV420P16.

The alpha plane can be removed using the RemoveAlphaPlane function. The following scripts can work.

input_m  = AvsFilterSource()    # YUVA444P16 (Y416)
input_m  = IsYUVA(input_m) ? RemoveAlphaPlane(input_m) : input_m    # YUVA -> YUV
input_m  = Is420(input_m)  ? input_m : ConvertToYUV420(input_m)     # YUV420
input_m8 = input_m.ConvertBits(8)    # YUV420P8 (YV12)

super    = SVSuper(input_m8, super_params)
vectors  = SVAnalyse(super, analyse_params, src=input_m8)
smooth   = SVSmoothFps(input_m, super, vectors, smoothfps_params, mt=threads, src=input_m)

Thank you for your great work!

[CrendKing]

Well, the new implementation does not need RemoveAlphaPlane anyways :). SVP is supported out of the box.

Added Y410: https://github.com/CrendKing/avisynth_filter/actions/runs/2189374110

Note that current implementation of Y41x do require you to have a CPU with at least SSE 4.1. AVX is not needed. I omitted the non-SIMD logic because I think whoever consumes 4:4:4 must at least have a relatively modern computer.

If both Y410 and Y416 work as you expect, you can close the ticket.

[BlackMickey]

Very good. Now I can use RGB filters like RIFE, Waifu2x, etc. and output in YUV444 format.

input_m = input_m.resize.Point(format=vs.RGBS, matrix_in_s="709")
smooth = core.rife.RIFE(input_m, model=rife_model, multiplier=rife_multiplier, gpu_id=rife_gpu, gpu_thread=rife_threads, tta=False, sc=True, list_gpu=False)
smooth = smooth.resize.Point(format=vs.YUV444P16, matrix_s="709")

[BlackMickey]

Why RGB32 is the preferred input format for YUV444 video? When RGB32 support is disabled, the preferred input format for YUV444 video is P210, not Y410.

[CrendKing]

@BlackMickey Can you provide steps, sample video and log to reproduce the issue? I'm seeing Y410.

Also, when adding comment on a long-closed thread, please tag me with @ so I may not see any notification.

[BlackMickey]

After testing, the source of the problem is MPCVideoDec. After resetting MPCVideoDec, the problem is restored.

I use MPCVideoDec because LAV doesn't support some special formats for hardware decoding, for example: YUV444P10 format.

[CrendKing]

Sorry, I can't test every decoder out there. So currently only focusing on LAV.

However, if you know a problem with alternate decoder and know how to fix, I'm happy to apply the patch, as long as no regression is introduced.