Transients for moving pictures processing

[DTL2020]

The most important part at the generating new transient from old samples values to new samples values (like fill border with constant value) is about transient shaping. Making sharp transient (zero sampes width) is typically worst case because typical moving pictures scalers (like sinc-based and jinc for 2D) will create large ringing on the total scaler kernel size.

From the japan ARIB STD-B28 Version 1.0 https://www.arib.or.jp/english/html/overview/doc/6-STD-B28v1_0-E1.pdf ​for transients design for colour bars - it is required to have from 6 to 9 samples encoding good enough quality transient (up to high-end broadcast quality transient for quality check).

A.5 Transient ​Ringing may occur when the stripe level of this color bar is suddenly changed, then this may possibly cause operational inconvenience. Therefore, it is necessary to carry out design while limiting bandwidths for leading edge and falling edge. ​The number of samples to be used for the transient shall be 6 to 9, in the case of 1920 horizontal samples, although it may depend upon the scale of hardware, process performance and the so-called “make up”.

In current AVS core the AddBorders also do not have any transients shaping. So it is required some workaround in scripting - cut transient area and apply directional blur or other filtering in some form and overlap back the processed transient area.

With external plugin it is expected some internal processing service to create good transient as single operation.

So for better plugin operation it is required new params like:

1. Filter type for transient shaping (typical applicable from AVS core: Gauss, UserDefined2, SinPow, Blur())
2. Width of transient in samples (default around 4..5, minimum 3). It may be even only number of samples if we expect center of new transient in-between last old (real) sample and first new (added) sample, so 2,4 and more. Or half-width in odd number from 1 or 2.

[Asd-g]

A.5 Transient you referred to is for a case with two solid colors and the transient area between them when the original content size is changed. I think this case is not relevant for this plugin. FillBorders is mainly meant for filling 1px line on any border of the image or padding. Depending on the used mode the filling is done by copying the next line (no transient area), by averaging the next 3 lines (no transient area). Actually mode=4 can benefit of such transient area but I haven't saw anyone using that mode. I will no bother (at least in foreseeable future) to implement transient area for this mode.

[DTL2020]

It is the case when user add new solid samples (expand frame like AddBorders) or fill some internals of frame (as FillBorders like to hide garbage and/or distortions at frame edges like at VHS captures).

The original moving pictures (video) frame samples have some math interconnections (Fourier spectrum limitation and shaping). The new added samples of solid colour have only required Fourier spectrum properties at long equal value running but the newly created transient area do not.

Example: Original conditioned frame samples of some 100% small peak: (16), 16, 27, 132, 235, 132, 27, 16, (16)

If user make FillBorders with black solid colour of 16 - (16), 16, 27, 132, 235, 16, 16, 16, (16) The newly created transient 235 to 16 is not conditioned to sinc(jinc)-based channel and will create large ringing on scaler (or short if scaler is of short kernel like Bicubic).

So the required task is to add newly created transient spatial shaping (typically by spatial filtering close to blur). The area of 16, 27, 132, 235, 16, 16, 16, must be processed with (optional) filtering depend on filter kernel size and set transient are width. The filtered result may be something like 16, 27, 132, 200, 50, 20, 16, - so samples values to both side of transient are changed.

On Tue, May 2, 2023 at 6:02 AM Asd-g @.***> wrote:

A.5 Transient you referred to is for a case with two solid colors and the transient area between them when the original content size is changed. I think this case is not relevant for this plugin. FillBorders is mainly meant for filling 1px line on any border of the image or padding. Depending on the used mode the filling is done by copying the next line (no transient area), by averaging the next 3 lines (no transient area). Actually mode=4 can benefit of such transient area but I haven't saw anyone using that mode. I will no bother (at least in foreseeable future) to implement transient area for this mode.

— Reply to this email directly, view it on GitHub https://github.com/Asd-g/AviSynth-FillBorders/issues/3#issuecomment-1530802019, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQMGLM7IWP3SVP4L2WX662DXEB2LBANCNFSM6AAAAAAXRUDDZA . You are receiving this because you authored the thread.Message ID: @.***>

[Asd-g]

You cannot have a result (16), 16, 27, 132, 235, 16, 16, 16, (16) from (16), 16, 27, 132, 235, 132, 27, 16, (16) using FillBorders.

Also Fillborders is for filling only borders of the frame not internals of the frame.

Please read again what's the purpose of the filter and/or test it yourself.

[DTL2020]

Hmm - I think mode 5: "fade" Fill the borders to constant value.

Is fill with constant value - but it really create sort of linear gradient to some colour. So linear gradient not create too much discontinuity for sinc-based scaler to ring.

But I anyway found mode with probability of creating too sharp transient: LoadPlugin("FillBorders.dll") ColorBars() PointResize(width4, height4) UserDefined2Resize(width/4, height/4, b=105, c=0) ConvertToYV12() FillBorders(50, mode=4) SincLin2Resize(width4, height4, taps=8)

As I see in mode 4 and with width of 50 it copies blue stripe to the left side and creates unfiltered transient from blue to white and it causes ringing at sinc-resize. So the probability of creating 'unconditioned' transient depends on samples values to warp from opposite sides of frame. If one size is black and other is white - it will create 'full-strike' sharp 'unconditioned' transient at appending of white samples to black samples without filtering.

Also it may be useful to add real 'fill only' mode like mode=7 to fill border with solid colour and also create nicely filtered transient to frame's content left. Currently AVS core designers also can not provide such functionality for internal LetterBox() filter. I open issue at AVS core development but there still no any progress - https://github.com/AviSynth/AviSynthPlus/issues/339

On Tue, May 2, 2023 at 9:08 PM Asd-g @.***> wrote:

You cannot have a result (16), 16, 27, 132, 235, 16, 16, 16, (16) from (16), 16, 27, 132, 235, 132, 27, 16, (16) using FillBorders.

Also Fillborders is for filling only borders of the frame not internals of the frame.

Please read again what's the purpose of the filter and/or test it yourself.

— Reply to this email directly, view it on GitHub https://github.com/Asd-g/AviSynth-FillBorders/issues/3#issuecomment-1531921825, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQMGLM3DPYGVNQGYGPK3VUDXEFEQNANCNFSM6AAAAAAXRUDDZA . You are receiving this because you authored the thread.Message ID: @.***>

[Asd-g]

Well, I already mentioned mode=4 in my first comment:

Actually mode=4 can benefit of such transient area but I haven't saw anyone using that mode. I will no bother (at least in foreseeable future) to implement transient area for this mode.

Filling with solid color... - as you mentioned, there are already filters doing that job.

[DTL2020]

" Filling with solid color... - as you mentioned, there are already filters doing that job."

But I still not know any plugin capable of creating good filtered transient while filling with solid colour. It is a sort of a shame for freeware video-processing software but it looks and ffmpeg and AVS core can not create correctly shaped transients for video even in 2023. I can understand it may be some issue at the very beginning of PC software development in 199x when PCs were slow and filtering may be slow too. But it looks old developers with understanding in video engineering is gone in 199x and 200x years and today both AVS and ffmpeg is abandoned software close to no one use it for video processing at home (only some companies for broadcast still try to use automation ffmpeg and AVS somewhere). But pro-usage of AVS and ffmpeg typically not good possible because it is old amateur-designed software with lots of quality issues. So either some complex scripting workarounds required (also with general AVS issues on RAM usage for complex scripting with lots of script objects and caching) and performance penalty. For example faster 2D blur for simple transient area selection in scripting is not best for quality - the better is select each transient direction (top/bottton and left/right) and apply 1D blur/filtering only. So if the transients filtering will be implemented it may be useful for both mode=4 and new fill-solid mode and also may be ported to AVS core later for LetterBox() and AddBorders(). May be as commit for that AVS core issue.

On Wed, May 3, 2023 at 6:00 AM Asd-g @.***> wrote:

Well, I already mentioned mode=4 in my first comment:

Actually mode=4 can benefit of such transient area but I haven't saw anyone using that mode. I will no bother (at least in foreseeable future) to implement transient area for this mode.

Filling with solid color... - as you mentioned, there are already filters doing that job.

— Reply to this email directly, view it on GitHub https://github.com/Asd-g/AviSynth-FillBorders/issues/3#issuecomment-1532402712, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQMGLMYKF4ISFOQ7OB4Z6MLXEHC4FANCNFSM6AAAAAAXRUDDZA . You are receiving this because you authored the thread.Message ID: @.***>

[Asd-g]

You can fill after resizing to avoid artifacts.

I will not spend time to add a mode for filling with solid colors w/ or w/o transients. PRs are welcome or you could wait for response in the issue you already opened in avs+ repo.

I leave this issue opened to remind me about mode=4.

[DTL2020]

" You can fill after resizing to avoid artifacts. "

It is not possible at many practical use cases. The digital moving pictures encoding in small frame size is also part of compression. Users typically like low frame size of very sharp images and low bitrate of MPEG and low file sizes for storage. So operating with very sharp digital moving pictures requires special care (and typical sinc-based scalers at display and DAC is typicaly sinc-based naturally too). Also to decode it to original analog form the display scaler is used (or DAC to make analog video). So the issue of unconditioned transient occur at end-user scaler (display or DAC) and need to be avoided at production stage. Typical use case is creating standard DVD PAL of 720x576 frame size from VHS analog capture. If we use some fill-borders to hide garbage at edges of lines and top and bottom lines and standard BT.601 sampling rate and the used software is of low quality (no proper shaping of newly created transients) - the user will get ringing at display or DAC of DVD-playback hardware at analog output. It is typically not possible to instruct display or DAC to make garbage-hiding after scaling.

On Thu, May 4, 2023 at 12:35 AM Asd-g @.***> wrote:

You can fill after resizing to avoid artifacts.

I will not spend time to add a mode for filling with solid colors w/ or w/o transients. PRs are welcome or you could wait for response in the issue you already opened in avs+ repo.

I leave this issue opened to remind me about mode=4.

— Reply to this email directly, view it on GitHub https://github.com/Asd-g/AviSynth-FillBorders/issues/3#issuecomment-1533783963, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQMGLM5OUKCQDEYAQXRP2ULXELFRFANCNFSM6AAAAAAXRUDDZA . You are receiving this because you authored the thread.Message ID: @.***>

[DTL2020]

I make example of simplest possible kernel (gauss) and only 1 new param (ts - transient size in half-size, so 0 (no processing),1,2,3,..) for transient filtering in pull-request of https://github.com/Asd-g/AviSynth-FillBorders/pull/4 . The left border processing looks like work OK for all widths (of left-fill param) and right still need some debug on low widths (about right=4 and ts=2). I think if you can re-design it in better C program it may not need to spent many time on right/top/bottom processing if you will redesign it in the possibly different way. The debug of handling of edge-conditions take many time with my poor programming skill. It takes me many hours to make and debug processing for left-only. The internals of frame processing far enough from frame edges is equal for all left/top/right/bottom transients and with equal kernel.

The no-resize filtering of transient with convolution is very simple but handling of edge-conditions add some complexity to function (the fetching must not run out of frame buffer and of edges of line/column, all out of border samples for convolution are copy of edge sample).

The gauss kernel param of 1.2f internal adjusted to have nice enough close to non-ringing transient at sincresize of a small with (to be sharp enough). May be also made of user-defined param if user like to adjust transient width to more blurry for example). If add more sharper kernels optionally (SinPow and UserDefined2) - the 2 more kernel adjusting params required as user-provided filter arguments.

Good enough results are with ts=2. The result of ts=1 is also much better in compare with nothing. So may be the C-convolution is fast enouth to compute. If not - the processing may be not very complex put to SIMD for better performance.