realfinder / AVS-Stuff

avs/avs+ Stuff
GNU General Public License v2.0
61 stars 15 forks source link

QTGMC: fft3d, dfttest and KNLMeansCL support native hibits #130

Open erazortt opened 3 years ago

erazortt commented 3 years ago

Instead of using the crappy lsb setting fft3d and dfttest could be used with native hibits content, which is much faster. There is even huge upside to use fft3d in high bits mode irrepective of the input clip bits, because this solves the banding issues fft3d has but is still faster then using dfttest on low bits. Suggestion would be to replace the lsbd variable with something more readable like "forceHighBitFFT" and change the calls like that:

inHighBits = input.BitsPerComponent()>8 lowInForceHigh = forceHighBitFFT && !inHighBits

(Denoiser == "dfttest") ? sneo_dfttest(lowInForceHigh ? noiseWindow.ConvertBits(16) : noiseWindow, ....) : \ (Denoiser == "KNLMeansCL") ? ... : \ sneo_FFT3D(lowInForceHigh ? noiseWindow.ConvertBits(16) : noiseWindow, ....)

dnwindow = (Denoiser != "KNLMeansCL" && lowInForceHigh ) ? dnWindow.ConvertBits(8,dither=1) : dnWindow dnwindow = (Denoiser == "KNLMeansCL" && lowInForceHigh ) ? dnWindow.ditherpost(....)

erazortt commented 3 years ago

Actually the newest versions of KNLMeansCL by printerf (v1.1.1d and newer) also support the native high bits. So my pseudo code from above would look like that: (Denoiser == "dfttest") ? sneo_dfttest(lowInForceHigh ? noiseWindow.ConvertBits(16) : noiseWindow, ....) : \ (Denoiser == "KNLMeansCL") ? SH_KNLMeansCL(lowInForceHigh ? noiseWindow.ConvertBits(16) : noiseWindow, ....) : \ sneo_FFT3D(lowInForceHigh ? noiseWindow.ConvertBits(16) : noiseWindow, ....)

nwindow = lowInForceHigh ? dnWindow.ConvertBits(8,dither=1) : dnWindow

realfinder commented 3 years ago

try https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.6%20and%20up/QTGMC.avsi

same for https://github.com/realfinder/AVS-Stuff/issues/129 and https://github.com/realfinder/AVS-Stuff/issues/128