[Enhancement] Improved program window scaling via super-sampled integer nearest neighbor

[NintendoManiac64]

The current "screen filtering" option seems to simply change the scaling algorithm from nearest neighbor to bilinear.

So since melonDS doesn't (yet?) have pixel scalers like scale2x (or the more intense xBRZ and/or hq2x scalers if you're into that) to help clean up aliasing on pixel art when doing non-integer nearest neighbor scaling, my proposal is doing what I believe mGBA does for its "bilinear (pixelated)" scaling mode on the 2DS/3DS - doing integer nearest neighbor scaling to whatever would be the next largest ratio would be that's larger than the current window size, and then downscaling to the program window via bilinear.

For example, on a 1360x768 display with the DS screens arranged horizontally with the program window maximized, 1360px is 2.65625x that of the 512px width of both DS screens side-by-side. That is... not a particularly high ratio, meaning non-integer nearest neighbor can look pretty bad at times.

With my proposal, you'd instead scale that 512px width to the next highest integer ratio which, in the case of a 1360x768 display, would be 1536px wide - an exact 3x scale. From there you would then downscale back to the display's 1360px width via a simple bilinear or the like.

(now to clarify, this doesn't have to be exclusive to having the screens in a landscape orientation or even with the two screens at the same size; it was just simpler for me to explain my proposal using the situation I have found myself in on a real life laptop I was recently using melonDS on)

Here's a visual example of what I mean, all scaled to 1360px wide; make sure to view the images in full:

======== Nearest neighbor (various diagonal slopes are ugly, like on the arrows) ========

======== Bilinear (blurry, 'nuff said) ========

======== Super-sampled integer nearest neighbor ========

[nadiaholmquist]

I'm very in favor of adding this, nearest prescale + bilinear is definitely the way to go for this because as you show, it retains the crispness of the image without causing visibly uneven pixel sizes. I might try and implement this.

[poudink]

I'd like it to be optional. I like my scaling unfiltered.

[nadiaholmquist]

Of course, the screen filtering option would be replaced with a submenu (or box in the video settings?) with options for nearest, bilinear and this new option (we could call it "sharp" or some other friendly name) so people who like the current options can keep those.

[NintendoManiac64]

If the nearest neighbor scaling gets revamped, then perhaps also a setting for emulating the pixel grid could also be implemented? By its nature it'd probably only work correctly when integer scaling is used.

Basically, at sufficiently high ratios (e.g. 500%), it can actually be more visually accurate to surround each of the DS's pixels with a black boarder to simulate the look of an LCD in a manner that's kind of like the LCD equivalent of scanlines (and I guess it could similarly be configurable with regards to how thick you want the aforementioned pixel grid - on a 4k+ resolution display you'd probably need the grid to be two pixels thick rather than just one)

Example image at 500% with single-pixel width grid (you'll definitely need to zoom it in to full size to see what I mean):

[poudink]

I get what you mean, but it seems like a pretty poor way to emulate the effect, IMO.

[memmam]

I was actually about to suggest this, except instead of scaling to the next integer scale and downscaling, you scale to the integer factor below that (e.g. if you wanted to scale both screens stacked vertically to fill 1920x2160, you'd have a scale factor of 5.625x, which becomes a flat 5x) and then bilinear scale UP the rest of the way. This is the method used on the RetroTINK 5X and 4K, and produces a VERY clean image. At some scales, the fact that it's bilinear scaled at all is almost imperceptible.

I'd be curious to compare the two methods and see what gives a better result.

Edit: Also, to follow on from the discussion above between @NintendoManiac64 and @poudink, a grid filter like that might actually be decent if it were semi-transparent rather than hard black pixels, again see the RetroTINK 4K, as well as the Analogue Pocket. Not sure how hard that'd be to implement.

[NintendoManiac64]

It's my impression that what people call "sharp bilinear" nowadays is exactly what I describe, going from the next largest integer factor and then scaling down.

I suggested doing it this way because, when I did testing, this way consistently gave the best results.

I imagine the reason why the RetroTINK does it the other way is because they're limited by maximum resolution and can't deal with going above whatever their maximum is due to being a hardware-based solution.

[memmam]

As someone who was involved in the testing and documentation of the RetroTINK-4K, I can tell you that the maximum practical resolution constraint is high enough that I don't think that was a consideration, with the device able to, for example, hit a 18x scale from 240p (resulting in a 4K window of an 8K image) without issue (this would be used for e.g. determining proper phase when creating a profile).

Out of curiosity, I actually tested our two methods, and the result looks virtually identical between the two. See for yourself:

[NintendoManiac64]

Use a smaller scale factor—the images in my examples are only around 2.6x which amplifies any differences.

Once you get to resolutions as high 8x+ the differences becomes indistinguishable; I mean, when you're turning 1 pixel into 8x8 or 9x9, that's almost to the point that you could even get away with plain standard non-integer nearest neighbor!

To clarify, the main difference is that there will be a sort of inconsistency and/or unevenness to the pixel anti-aliasing if you scale up with bilinear rather than scale down (kind of like if you did direct nearest neighbor, but to a much lesser degree). I recalled this being the case when I originally submitted this issue and I just confirmed right now that it's still the case.

It can be a subtle difference that requires you to zoom in anyway on modern high-res screens but, despite this issue having been created in 2021, it's actually something I had mentioned in the comments many years earlier—back when 768p screens were commonplace on both TVs and laptops which is why I used a horizontal resolution of 1360px.

[memmam]

The reason I scaled to 5.625x is because it covers a common use case, scaling a single DS screen to 1080p.

Here's the same test using 720p as a target:

Your method looks sharper under this test, and additionally that if you're scaling both DS screens (384 pixels tall) to 720p, you get a scale factor of 1.875x, where downscaling from 2x makes more sense. Additionally, the RetroTINK use case is based on the assumption you're scaling to at least 4x nearest neighbor to start, which mitigates the effect some.

Additionally, upon further reflection I may be misremembering and it may be nearest neighbor scaling to the closest integer factor, and then scaling up or down as appropriate. I'll need to check with Mike.