haasn / libplacebo

Official mirror of libplacebo
http://libplacebo.org/
GNU Lesser General Public License v2.1
547 stars 69 forks source link

filters: Add Power of Blackman and Power of Garamond windows #220

Open garamond13 opened 10 months ago

garamond13 commented 10 months ago

Libplacebo supports 2 free parameters for windows and in my research (https://github.com/garamond13/Finding-the-best-methods-for-image-scaling) more customizable windows have proven to be superior in quality when scaling images. Also compared to Said window and GNW, for these 2 windows we don't need to do any extra modifications to the code in order to implement them.

Reference for Power of Blackman window: https://github.com/garamond13/power-of-blackman

Reference for Power of Garamond window: https://github.com/garamond13/power-of-garamond-window

haasn commented 10 months ago

Just a random thought, but if it truly is useful to take the power of a window in a more general context, it would also be possible to just add it to pl_filter_config. This would make using it a bit harder in practice (basically requiring scale=custom), but would provide more flexibility.

From a mathematical PoV, what effect does taking the power of a window have on its frequency response?

garamond13 commented 10 months ago

Just a random thought, but if it truly is useful to take the power of a window in a more general context, it would also be possible to just add it to pl_filter_config. This would make using it a bit harder in practice (basically requiring scale=custom), but would provide more flexibility.

From a mathematical PoV, what effect does taking the power of a window have on its frequency response?

It would basically control the main lobe width. I don't think that it would be very useful for all the other windows.

haasn commented 10 months ago

Looks good to me now, can you squash your commits into two? (One commit to add garamond window, one commit to change blackman window)

garamond13 commented 10 months ago

Looks good to me now, can you squash your commits into two? (One commit to add garamond window, one commit to change blackman window)

Done, I think.

estrogently commented 10 months ago

I'm working on my own script for comparing the performance of windowing functions across a dataset, I'll check how much the additional parameter helps Blackman and how well Garamond performs and post the results back here in a day or so a few hours from this edit.

estrogently commented 10 months ago

Both the powers in Garamond and the power added to Blackman should always be nonnegative, else they produce very silly windows. Additionally, if 0 < Blackman's power < 1, it NaNs for most of the window's radius, and if the inner Garamond power = 0, it zeros the whole filter.

For my testing, I'm using a derivative of the RealSRv3 dataset (I'd also like to try with Manga109 but I'm waiting on access). I'm testing them as windows for ewa_lanczos with 3, 4, and 5 lobes. A derivative-free optimization algorithm picks the antiringing, sigmoid center, and window parameters. From those I use Robidoux's criteria to derive the blur[^1] and sigmoid slope[^2]. The LR dataset is then enlarged with vf_libplacebo, and Butteraugli scores are computed of the enlarged images vs their HR ground truth counterparts. The mean score is then returned to the optimization algorithm, and the optimal score found is taken as the score for that combination of kernel + window + lobes.

Compared to the other adjustable filters, Garamond scores slightly better than Kaiser at 3 and 5 lobes with 0.70% and 0.17% lower scores respectively. At 4 lobes, original Blackman beats it with a 0.16% lower score. The optimal parameters for Garamond seem pretty unstable between lobe counts (with other windows the best parameters for e.g. 3 and 5 lobes are fairly close). Note the additional parameter does result in more overfitting in the optimization.

I wasn't able to obtain results for Power of Blackman. For some reason the resulting function was ill-behaved enough the integration just completely stalled out, which is a first for the windows I've tested. I suspect it can probably get slightly better results than the original Blackman (an extra parameter will just always let it overfit better), though you can't raise the power much before it starts zeroing everything past 20-40% of its radius.

[^1]: One run does "sharp", another "sharpest" [^2]: Enlarge a single white pixel on a black background infinitely using the filter, and pick sigmoidization parameters so that the average luminance remains the same as before enlarging. I have implemented this using numerical integration

haasn commented 10 months ago

@estrogently it sounds like we should leave original blackman untouched, and simply add the tunable garamond window?

estrogently commented 10 months ago

@estrogently it sounds like we should leave original blackman untouched, and simply add the tunable garamond window?

that sounds good, I'd also add an fmax with 0.0 to both garamond tunables

garamond13 commented 10 months ago

@estrogently could you please retest Power of Blackman with alpha in range [-inf, 0.16]?

For Power of Garamond we should return 1 if n is 0.

estrogently commented 10 months ago

I capped the alpha at 0.16 and let it run overnight. At 3 and 4 jinc lobes the power converged to 1, at 5 lobes it didn't and did score better than original Blackman, though not as well as Kaiser or Garamond

garamond13 commented 10 months ago

Power of Blackman may not be defined for all x if alpha > 0.16. This was probably causing issues before to @estrogently when doing tests. If you're ok with this solution I can squash this into 2 commits.