Added pmode=1

[DTL2020]

With C-ref and AVX2 processing functions. Support 8 and 16 bit samples.

[DTL2020]

I tried to change indent to 4 at all text of .cpp and _AVX2.cpp file and remove all unused lines (commented out and unused functions). Result is in the commit of https://github.com/Asd-g/AviSynth-vsTTempSmooth/pull/8/commits/c77415ba69f8aa71e3c18f0955da8b67154f6985 .

Also it looks any IIR-type filtering (with using memory of previous frames) in AVS+ and multithreading modes are only possible with MT_SERIALIZED cache mode. So I tried to add this cachehints selection if pmode=1 and any IIR-type processing enabled (taken from mvtools2 - https://github.com/DTL2020/mvtools/blob/f4b30d1ea2d9a7e3ec13808beb68c7dbf349ae23/Sources/MVAnalyse.h#L162 ).

[DTL2020]

Everything is done with commits up to https://github.com/Asd-g/AviSynth-vsTTempSmooth/pull/8/commits/dd5b71a814e8aa40eac7fea5c03b1b72cd812fb4

[Asd-g]

The last thing is the indentation. You do https://github.com/Asd-g/AviSynth-vsTTempSmooth/commit/c77415ba69f8aa71e3c18f0955da8b67154f6985 but that didn't fix the issue (only the changes aren't visible).

I created patches for your commits except the latest two. For the latest commits you can create patches (git format-patch c77415ba69f8aa71e3c18f0955da8b67154f6985..dd5b71a814e8aa40eac7fea5c03b1b72cd812fb4). You have to hard reset to 77400e19d990ff7251c99b24af8e7d4bda644692 (git reset --hard 77400e19d990ff7251c99b24af8e7d4bda644692). Then apply one patch 0xy...patch (git apply 0xy...patch). Then commit the changes from the patch (git commit -a -m "....."). Repeat above two steps for every patch. Then apply the latest two patches (git am 0001...). Finally you have to force push (git push -f).

005_Fixed current and ref.patch 006_Fixed SumMem init for.patch 007_Removed comments.patch 008_Added params for.patch 009_Added usage of pnew.patch 010_Redesigned MEL to.patch 011_Added AVX2 (partial).patch 012_Added 16bit samples support for pmode=1 into.patch 013_Started full-SIMD pmode=1 for.patch 014_Replaced srcp with pfp at.patch 015_Finished full-SIMD.patch 016_Ready for PR with.patch 017_Removed commented out.patch 019_Changed indent to.patch 020_Removed all commented out.patch 018_Fixed some places.patch 001_Added pmode=1.patch 002_Added IIR mode.patch 003_Added memory of previous.patch 004_Fixed mem init.patch

[DTL2020]

Oh - I really do not know anything about command line git and only know to use web-github (and sometime use github-offline but it sometime fail to send commits to github-web and I stop use it). May be somehow the last issue with indenting can be fixed using Visual Studio 2019 interface ? I tried to set options and ctrl+k and ctrl+d and save all (rebuild). May be I need to select all text ctrl+a at first ?

[Asd-g]

Tomorrow I cherry-pick all the commits ignoring the whitespaces.

[DTL2020]

So in 2023 we do not have any working text editor tools to fix some auto-formatting with leading spaces added by Visual Studio ? Or it is some non-compatibility between Visual Studio text editor and git/github sources compare tools to display only changed lines ? Are there any recommendations for the future how I can avoid this issue when making fork at github-web and using Visual Studio 2017/2019 as text editor ? So that the will be backward-compatible for pull-request with required lines formatting. I need to find used indentation settings (or ask author ?) in the project and set manually in the Visual Studio options after opening project (or add .editorconfig file with correct settings for auto-applying) ?

On Fri, Jun 30, 2023 at 9:48 PM Asd-g @.***> wrote:

Tomorrow I cherry-pick all the commits ignoring the whitespaces.

— Reply to this email directly, view it on GitHub https://github.com/Asd-g/AviSynth-vsTTempSmooth/pull/8#issuecomment-1615065286, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQMGLM2YF4KAW5GR6UHSLL3XN4NODANCNFSM6AAAAAAZTMZXXQ . You are receiving this because you authored the thread.Message ID: @.***>

[Asd-g]

So in 2023 we do not have any working text editor tools to fix some auto-formatting with leading spaces added by Visual Studio ? Or it is some non-compatibility between Visual Studio text editor and git/github sources compare tools to display only changed lines ?

It's about the git history. You did already commits with different indentation. The git history containing this commit should be rewritten. For example this commit https://github.com/DTL2020/AviSynth-vsTTempSmooth/commit/f31e1615fbc40897485aac49fe5c08893052072c must be rewritten like this one https://github.com/Asd-g/AviSynth-vsTTempSmooth/commit/45558a26b6f36e9a33824ac30092f29a08991d4e (note src/vsTTempSmooth.cpp).

Are there any recommendations for the future how I can avoid this issue when making fork at github-web and using Visual Studio 2017/2019 as text editor ? So that the will be backward-compatible for pull-request with required lines formatting.

Changing Visual Studio defaults is probably the best thing to do (Tools->Options->Text Editor->C/C++->Tabs->Tab size=4 Indent size=4, Insert spaces). Or as minimum use that editorconfig file but be aware of effects from Ctrl+K, Ctrl+E/Ctrl+K, Ctrl+D (info).

I need to find used indentation settings (or ask author ?) in the project and set manually in the Visual Studio options after opening project (or add .editorconfig file with correct settings for auto-applying) ?

I think the majority uses tab size 4 with spaces. If you do one commit and notice that every indentation is changed you should know that you're using different indentation settings.

[DTL2020]

"It's about the git history. You did already commits with different indentation. The git history containing this commit should be rewritten."

May be if it is something complex about git history (with many commits at development time) it is better to delete my repository - make new fork of your repository and make one only commit of all new and changed files (after applying indentation of 4/space) ? So the git history will be clear from old commits with other (Visual Studio default) indentation settings ?

"I think the majority uses tab size 4 with spaces."

To solve something like this issue with indentation, pinterf made (for me) .editorconfig file for mvtools2 project - https://github.com/DTL2020/mvtools/blob/mvtools-pfmod/.editorconfig

And I see it uses [*.{cpp,h,hpp,c,asm,hlsl}] indent_style = space indent_size = 2 trim_trailing_whitespace = true end_of_line = crlf insert_final_newline = true

So it looks even AVS plugins of different developers uses different indentation settings for C/C++ files ?

[Asd-g]

May be if it is something complex about git history (with many commits at development time) it is better to delete my repository - make new fork of your repository and make one only commit of all new and changed files (after applying indentation of 4/space) ? So the git history will be clear from old commits with other (Visual Studio default) indentation settings ?

As you probably already noted I cherry-picked almost every commit. Also there is cherry picked from commit ... in the message of the every commit.

Can you write a description of pmode=1, y/u/vthupd and y,u,vpnew so I can finish the documentation?

So it looks even AVS plugins of different developers uses different indentation settings for C/C++ files ?

Apparently. He uses also spaces but only 2 instead 4.

[DTL2020]

" a description of pmode=1, y/u/vthupd and y,u,vpnew so I can finish the documentation? "

pmode=1 - 'statistical' processing mode. It tries to select the 'best' sample from the total pool of input samples in the temporal axis for current position in frame using some algorithm of selecting 'most equal/probable' value. So it does not compute new sample values (not existent in input frames). With enough temporal radius it may output a more stable sample sequence duplicating 'best' value in several frames. With thupd=0 it work as FIR-type filter (no memory used and can support full quality seeking operations also any multithreading mode in AVS+). The selected to output 'best' value difference from current input sample value limited by y/u/vthresh param (same as in pmode=0). If difference above threshold - the input sample used for output.

y/u/vthupd - IIR memory update threshold. If y/u/vthupd > 0 - enable IIR-type filtering. It remembers previous 'best' sample output and compares current 'best' with memory value. If the difference below thupd - the memory sample used for output. It allows the creation of a more stable output sequence (less temporal noise and more MPEG-friendly) but may add artifacts on really moving or other way changing areas. It looks like any AVS filter with memory of reference previous frames values can only work correctly in MT_SERIALIZED AVS+ MT mode. So the filter tries to set this mode at startup if y/u/vthupd > 0.

y/u/vpnew - penalty for the new 'best' sample update in the IIR-memory. It is added to the internal metric of the current 'best' sample in the algorithm of replacing the current stored value with new 'best'. Allow to decrease the number of memory replacement operations and create more stable temporal output. Close to idea of penalty to update current best motion vector with new candidate with a bit better metric in MAnalyse from mvtools.

Values of y/u/vthupd and y/u/vpnew internally auto-scaled for integer bitdepth > 8 proportionally (same as Ythresh).

Addition: maxr param in pmode=1 may be in range 1..128.

Addition 2: Also pmode=1 may be used as general-purpose majority processing for any (up to 128*2+1) odd-number sources: 3 sources example: For input clips a,b,c: Interleave(a,b,c) vsTTempSmooth(pmode=1, Ythresh=255, maxr=1) SelectEvery(3,1) It will output 'voting' result of all sources for each sample for 'best' (as most probable between all analysed sources) sample value. For example if a,b,c clips recorded by 3 different receivers with different distortions at different time (frame or position in frame) and generally only 1 of 3 clips is damaged at the same time (frame of place in frame) - it will auto-select 'good' samples from other 2 clips.

On Sat, Jul 1, 2023 at 8:33 PM Asd-g @.***> wrote:

May be if it is something complex about git history (with many commits at development time) it is better to delete my repository - make new fork of your repository and make one only commit of all new and changed files (after applying indentation of 4/space) ? So the git history will be clear from old commits with other (Visual Studio default) indentation settings ?

As you probably already noted I cherry-picked almost every commit. Also there is cherry picked from commit ... in the message of the every commit.

Can you write a description of pmode=1, y/u/vthupd and y,u,vpnew so I can finish the documentation?

So it looks even AVS plugins of different developers uses different indentation settings for C/C++ files ?

Apparently. He uses also spaces but only 2 instead 4.

— Reply to this email directly, view it on GitHub https://github.com/Asd-g/AviSynth-vsTTempSmooth/pull/8#issuecomment-1616013001, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQMGLMZS32CES3QPKKRFIWTXOBNNHANCNFSM6AAAAAAZTMZXXQ . You are receiving this because you authored the thread.Message ID: @.***>

[DTL2020]

In the commits up to https://github.com/DTL2020/AviSynth-vsTTempSmooth/commit/1eb93d909ea9d187e7f47d27be808625fe5c9e3d I added also float32 samples processing in C-ref and AVX2 functions.

[DTL2020]

Trying to found what cause sometime crash of VirtualDub with some frame widths in float32 processing I found it looks AVS+ provide mod64 frame buffer stride only in bytes. But not in samples. If ComponentSize > 1 byte it can cause non-mod64 frame buffer stride. So 720 width frame is 2880 bytes stride (it is mod64) but if AVX2 implementation process 32 samples of 4 bytes each per pass - it finally cause buffer overrun because 720 is not mod64. So it looks all AVX2 functions with good enough 'workunit size' for pmode=1 can not be designed with single full-frame scan with single nested loops. It is required lower sized prologue after main processing of 32 samples per single SIMD pass. May be prologue is enough to be 16 float32 samples processing so 16x4=64 bytes ?

[DTL2020]

After looking into 'median' algorithm - it looks the sample-based implementation of pmode=1 is close or equal to 'median' value calculation. But C-algorithm is slow enough (typical median as I see calculated via sorting of list of values). But in SIMD implementation it become usable enough (using extreme compute power of SIMD to brute-force single pass computing and selecting output value using also SIMD-gather operation).

[Asd-g]

Closing this. Open a new pr for further changes.