Confusion about supported color spaces,...

[Selur]

Small question for clarification: What color spaces are supported when using Vapoursynth? Because when loading an 8bit 4:2:0 source, I get no problem, but when loading a 10bit 4:2:0 source I get 'only YUV444P10 and RGB30 are supported!' So does KNLMeansCL only support 4:2:0 when using 8bit? script I used:

import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/KNLMeansCL/KNLMeansCL.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Loading E:\Output\10bitTest.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="E:/Output/10BITT~1.MKV", format="YUV420P10", cache=0)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25/1
clip = core.std.AssumeFPS(clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# denoising using KNLMeansCL
clip = core.knlm.KNLMeansCL(clip=clip, device_type="gpu", device_id=0)
# adjusting output color from: YUV420P10 to YUV420P8 for x264Model (i420)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited")
# Output
clip.set_output()

also adding:

clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P10, range_s="limited")

doesn't seem to fix the issue, when using:

# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/KNLMeansCL/KNLMeansCL.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Loading E:\Output\10bitTest.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="E:/Output/10BITT~1.MKV", format="YUV420P10", cache=0)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25/1
clip = core.std.AssumeFPS(clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# denoising using KNLMeansCL (bit depth: 10)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P10, range_s="limited")
clip = core.knlm.KNLMeansCL(clip=clip, device_type="gpu", device_id=0)
# adjusting output color from: YUV444P10 to YUV420P8 for x264Model (i420)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited")
# Output
clip.set_output()

I still get:

Python exception: knlm.KNLMeansCL: only YUV444P10 and RGB30 are supported!

using:

clip = core.resize.Bicubic(clip=clip, format=vs.RGB30, matrix_in_s="470bg", range_s="limited")

does the trick for me, so from the looks of it bitdepth > 8 bit requires a conversion to RGB30.

-> not sure whether this is a bug or I just misunderstood something :)

[Khanattila]

Hi Selur, 10-bit precision with YUV420 is just a waste of computing power. Instead, promote the video to a 16-bit bit precision and work with that.

[theChaosCoder]

So the msg should be "only YUV444P16 and RGB30 are supported!" ?

[Khanattila]

Not at all, YUV444P10 is a special case if you want to denoise Y and UV together (channels = "YUV"). Benchmark

[Selur]

Sorry, but I still don't get it. In my example YUV444P10 doesn't work and prompts me with:

knlm.KNLMeansCL: only YUV444P10 and RGB30 are supported!

using YUV444P10 I get the same message,... YUV444P16 and RGB30 both do work, so if YUV444P10 and YUV420P10 should work and just be slower, where to I go wrong?

[4re]

Because the default for channels is "Y", try with channels="YUV". For P16 it should be the same though...

[Selur]

I can confirm that using

clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P10, range_s="limited")
clip = core.knlm.KNLMeansCL(clip=clip, device_type="gpu", device_id=0,channels="YUV")

works, but I don't understand what channels is for then. thought it was to select the channel that is used and a YUV444P10 source should have a 'Y' channel that could be filtered,.... According to the description:

Set the colour channels to be denoised. Possible values are "YUV", "Y", "UV" or "RGB", depending on the source colour format. By default, "Y" is denoised if colour space is YUV, otherwise "RGB".

'Y' for YUV...-content should be fine should. -> still totally confused

[Khanattila]

I invite you to read more on the subject, especially on the Y'UV etc colour model. Otherwise, it becomes difficult to understand the operation of the channels parameter.

The easiest and fastest thing is to promote a video from the YUV420P10 to the YUV420P16 and work with it. Then if you prefer you can go back to 10-bit.

// 10-bit video input
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P16, range_s="limited")
clip = core.knlm.KNLMeansCL(clip=clip, device_type="gpu", device_id=0, channels="Y")

[gundamftw]

The problem with this is that the error message should be more specific, like specifically tell the user to set channels to "YUV" if he is passing in a YUV444P10 clip.

[End-of-Eternity]

I just came across this error again when denoising a GRAY clip because I'd forgotten to increase the bitdepth to 16 bit from a 10 bit source. This made debugging a nightmare because I spent about 10 minutes trying to work out how my clip was still in YUV or RGB colourspace when I had already extracted each plane from the original clip individually. Once I realised, "oh wait, this is in 10 bit, that's probably why it's getting annoyed," it made total sense.

I'd suggest that the error message should include the format of the input clip telling the user that subsampled or individual 10 bit planes are not supported, to make this process a whole lot less confusing. If I'm not too busy later today I'll submit a quick PR for this. Cheers