Majority processing mode (new mode)

[DTL2020]

Close to the currently implemented on block-basis in MDegrainN but here on per-sample basis: (for C-implementation see 2 loops from https://github.com/DTL2020/mvtools/blob/a1989c8fc9092e7c3f8713dc19241a3bef20ab00/Sources/MDegrainN.cpp#L6892 )

For each set of samples in tr-pool calculate 2D difference table of each sample with each sample and calculate sum of each row. Next search for lowest sum row index. Next is select to output the sample from tr-pool selected by this index.

It is 'hard' selection mode is simplest. But may be supplemented with some weighted averaging processing to include in output result more samples with lower weight in compare with 'best' (lowest sum) using some weighting function based on the sum value (lowest sum gives largest weight and highest sum give lowest weight).

It is close to the old plugin described in https://forum.doom9.org/showthread.php?t=78338 but work with any size of tr-pool. Not tr of 1 only (2*1+1 = 3 input samples to analyse).

May be used as secondary of final stage of denoising to get more stable and MPEG-friendly output.

Also may be supplemented with complete IIR-filter activity using previous-frame-sample_and_hold-memory (data buffer of one input frame-size only so small enough) and update thresholding (input param) (also see that MDegrainN complete function for implementation). So it will be majority-based IIR temporal filter with non-linear activity for possible 100% temporal denoise after getting low enough input noise levels.

For per-sample processing may be other better algorithm exist to select most-probable value from 1D vector of values. May be better formula: value most close to most probable in contigous values dispersion in the input 1D array. Or rounded to output precision most probable value (may be alternative optional mode if implemented).

To more safely workaround of real scene transforms and scenechanges - only samples with difference below user-provided threshold accepted to analysis. Other samples got 'max possible difference' value in table.

[Asd-g]

No eta. PRs are welcome.

[DTL2020]

Visual Studio 2019 can not build current project: it can not load AdvancedVectorExtensions512 string from build config.

Adding /arch:AVX512 command line string to custom build of _AVX512.cpp file solves issue and executable can be built. (I think it is not safe to add /arch:AVX512 to all project build options because it may be unsafe for pre-AVX512 machines ?).

Can you change solution files if /arch:AVX512 command line switch is compatible with your build environment too ?

Without adding /arch:AVX512 - the VCL do not provide required headers to compiler and it fails with 'can not found zero_si512() function' error.

[DTL2020]

Well - this commit https://github.com/DTL2020/AviSynth-vsTTempSmooth/commit/f31e1615fbc40897485aac49fe5c08893052072c shows how it can look like. It have working build to test https://github.com/DTL2020/AviSynth-vsTTempSmooth/releases/tag/1.3.0_a.01 . The new function filterI_mode2() added. Still FIR-type only (no memory of previous frame output). It looks I somehow lost with complex templating and can not make nice HBD-templated version. Currently only 8bit is implemented. It is slow (slower in compare with C-ref of standard mode). It mostly probably can be also put to several samples processing in SIMD but need some thinking how. For MDegrainN it was not designed because the block dismetric processing is much slower in compare of DM-table calculation. Also the number of blocks in mvtools is much less in compare with number of samples in frame. Here dismetric is only abs difference of 2 numbers and can be put to SIMD I hope. Will try to add 3 planes memory buffers and new params 'th_upd' later for IIR-type processing (currently commented out in function - you can see how it looks) later.

[Asd-g]

Updated the msvc files. I hope they are ok now for VS2019 now.

[DTL2020]

Some general programming idea:

Currently startup selector of main processing function using if-else-switch blocks become very complex and not nice to use. With addition of new processing mode it become even less nice to use because it require sort of doubling size of conditions combinations. May be you can re-write multi-argument processing function selector to different way used by pinterf in mvtools. Example is: https://github.com/DTL2020/mvtools/blob/0a6b093507bc757457783d537bc6cfaaa273989d/Sources/PlaneOfBlocks.cpp#L6538

It looks use some standard c library classes std::map and std::tuple. And allow to easy select required processing function of a very big number of possible initial params in any combination and also make fallback to default (c-ref) if no matching function exist for provided list of params.

Other example with more complex search is https://github.com/DTL2020/mvtools/blob/0a6b093507bc757457783d537bc6cfaaa273989d/Sources/SSIMFunctions.cpp#L155

[DTL2020]

Also some idea about AVX512 building with VisualStudio - typically it not any critical to intrinsics used and can nicely build SSE2 default project with any .cpp file containing any supported SIMD family intrinsincs (up to AVX512 with many of 201x versions of Visual Studio, at least 2017 and 2019 can build up to AVX512 without any special command line or project settings from defaults, with simple immintrin.h included). No special handmade /arch:AVX512 command line switch required.

But it looks VCL intermediate library have some protection from high-SIMD building if not some 'defines' provided by compiler. It looks the /arch:AVX512 command line switch also forces some internal defines in C-preprocessor so VCL can provide required wrapper functions to compiler. So may be required defines may be provided in _AVX512.h or _AVX512.cpp file before including VCL headers and it will auto-switch VCL to AVX512 without additional hacks around compiler options ?

[DTL2020]

Finally first pull-request is ready - https://github.com/Asd-g/AviSynth-vsTTempSmooth/pull/8 . With AVX2 version running about 30x times faster in compare with C-ref it is about usable with not very big maxr values (about 10..20).

[DTL2020]

After some more reading into different average math methods: I think the documentation may be updated with next text:

pmode=1 processing is Medoid value (https://en.wikipedia.org/wiki/Medoid ) and uses absolute samples difference as dissimilarity metric for 1D objects like numbers. And in this case it is equal to median value. So with thresholds = 255 and no IIR-processing it is 99% or more equal to MedianBlurTemporal AVS plugin. It is slower with maxr=1 and possibly 2 and significantly faster with maxr > 2. Also it provides additional functions of possible threshold to limit max difference with source sample and some optional temporal IIR filtering.