mickleness / pumpernickel

This Java project includes classes related to desktop applications, Swing, performance, image processing, data structures, and other misc topics.
https://mickleness.github.io/pumpernickel/
MIT License
66 stars 12 forks source link

Scaling: Review scaling, esp relating to upsampling document icons #82

Closed mickleness closed 1 year ago

mickleness commented 2 years ago

Reviewing this PR reminded me of my Scaling classes.

I checked how they would perform compared against bicubic interpolation.

If I take a 16x16 icon and scale it to 24x24, bicubic interpolation renders it as:

image

(this is a real-world scenario where the primary monitor is set to 100%, and then you drag the window to a monitor with a resolution 150%.)

The Scaling class renders this as:

image

So right away there appears to strange artifacts (maybe a bug?) in the bicubic rendering

Meanwhile the Scaling implementation is clearly messing up the bottom row of pixels. (And it's probably (?) also messing up the right-most pixels too? I assume it's doing something similar on the bottom and right edges, but the righmost pixels of this 16x16 icon are transparent.)

So this ticket is to fix the edges of the Scaling class's rendering, and to generally review performance and see if the Scaling class offers a competitive edge. (I have no idea what direction the original JDK bug / PR will take, or what my timeline for this ticket is. So I'm working on this for its own sake, and it may end up not relating to that JDK ticket at all.)

To resolve this ticket I need to:

For reference bilinear interpolation looks like: image

nearest neighbor looks like: image

mickleness commented 2 years ago

Performance-wise: I just pushed a commit that significantly improved performance. (Now when you scale this a 48x48 icon it executes in about 42% of the time the bicubic model requires. Previously it was around 75%.)

There's probably still room for further improvement, but I need to wrap this up for tonight:

mickleness commented 2 years ago

As of the last commit the scaled 24x24 icon resembles:

Screen Shot 2022-03-25 at 4 21 12 AM

mickleness commented 2 years ago

If we rotate this so the source 16x16 image has a black pixel on the left and right, then the 24x24 image renders as:

image

This is not perfect (the second column should probably also be a shade of gray), but I consider it good enough for now.

mickleness commented 2 years ago

Update: I double-checked: this will not be a trivial fix. Currently we map a row of source pixels to dest pixels, and in this case (upsampling a 16px row to a 24px row) that map resembles: [0, 1, 3, 4, 6, 7, 9, 10, 12, 13, 15, 16, 18, 19, 21, 23]

If a dest index is not listed here: then its value is interpolated. So in this case: at x=0 and x=1 the exact value of the src is copied, which means there isn't any interpolation.

IIRC this model was originally designed for fast downsampling; upsampling was added as an afterthought. This use case shows we'd need a bigger refactor to improve it. (And I'm not submitting that as a ticket here because I don't have a pressing use case for that currently.)