[3DS] Distorted Image With Bilinear Filtering is Disabled

[saphtea]

Description

The image becomes distorted when disabling the bilinear filter and disabling integer scale. This issue does not happen with integer scale enabled.

This is a bit of a problem though because desirably I'd like a pixel perfect scale to my screen. Integer scale doesn't provide that.

Expected behavior

Once bilinear filtering is disabled I expect it to be scaling the screen via nearest neighbor without a loss of quality/pixels.

Actual behavior

When disabling the bilinear filter scaling seems to happen via nearest neighbor, but there is a very obvious loss of quality and in general there seems to to be a loss of pixels being displayed.

Steps to reproduce the bug

1. Launch a game (I've used Kirby's Dreamland 2 in Gambette and Mother 3 in gPSP)
2. Go to Video Settings
3. Disable Bilnear Filtering
4. Disable Integer Scale

Bisect Results

This has happened with pretty much every version I've used

Version/Commit

You can find this information under Information/System Information

• RetroArch: [version/commit]

Version 1.6.0

Hardware: New 3DS XL Firmware: 11.4.0-37U

With Integer Scale and Bilinear Off:

With Integer Scale On and Bilinear Off:

The second photo is how the first settings should look except they should be scaled up without any introduced artifacts or blurriness.

EDIT: It looks like this feature request here: https://github.com/libretro/RetroArch/issues/2846 would probably fit well to fix this issue or maybe just replace bilnears default functionality to do this as the feature request talks about.

[ghost]

I noticed the same problem but not sure what is going on, there may be a precision issue with the scaling code. Perhaps @danieljg has an idea?

[danieljg]

I've been having a look at this for a couple days now, as it's quite noticeable, but I haven't figured out what's wrong yet. Another simple way of seeing this effect in action is to simply run a CPS2 game (such as street fighter alpha) in final burn alpha, then turn on the 'diagnostics' option and go to the 'dot cross hatch' screen.

It definitely seems like a precision issue, like something was being truncated or else decided with garbage bits in the scaling algorithm, like z-fighting, but on just two dimensions. I still don't understand quite how the scaling works, but the 'scale_vector' that maps the viewport coordinates to the framebuffer coordinates seems like a prime suspect. The scaling algorithm is funky in general when integer scaling is specified along with an aspect ratio which won't fit, and simply chooses to fill the screen.

Also, I'm glad this is filed as major, it always surprised me that no one else seemed to notice the issue on the web. Another thing no one else seems to mention is that, when setting the slider to a little below 50%, only the left eye image is rendered.

I will keep having a look at this, maybe if I hack at it enough I'll crack it.

[hizzlekizzle]

This looks like normal non-integer NN scaling to me. It's very noticeable at low scale factors because a single pixel going one way or the other is more obvious at, say, 1.8x than it is at, e.g., 4.5x.

[ghost]

@hizzlekizzle may be right. The same problem exists in my VGA driver for RetroArch on DOS which also uses NN scaling.

Here, the original 320x240 RGUI menu is scaled down to 320x200. Or notice a 256x224 NES game scaled up to 320x200, particularly the word Nintendo.

We did know about the 3D slider issue as well but just nobody has gotten around to fixing it.

[danieljg]

This seems like something the -fast-math option would produce. Typically, there shouldn't be nearly as much row and column fighting as the 3ds core shows. It's quite extreme there, although the screenshots by Saphiresurf don't show it as clearly.

I'll try to get some pictures later this afternoon, and maybe compile without the fast-math optimization to try things out.

[saphtea]

Updated the pictures to compare Mother 3 in gPSP. The difference is a bit more dramatic and pronounced. Tell me if there's anything else needed here!

[andres-asm]

hmmm nearest at such low resolution is just like that...

[danieljg]

how do you take screenshots those screenshots in the 3ds version? my camera is not in very good shape, though i can borrow my wife's if needed...

[hizzlekizzle]

Yeah, this indeed appears to be low-precision in the texel sampling paired with NN scaling. I'm not sure how we would go about raising the precision, but another option might be to pre-scale the image with NN at an integer scale (either larger than the screen or smaller) and then up/downsample from there with bilinear.

[saphtea]

I think hizzlekizzle may have the right idea as I don't think NN is actually possible at non-integer scales (that I know of). I've been looking through a few different emulators for the 3DS and any emulator that claims a "pixel perfect fit" seems to do it with bilinear (as they're all blurry). There's also some of Nintendo's official stuff that we can try to use to understand more about this.

Well, really what we have is AGB_FIRM which is the 3DS' firmware for running GBA games, TWL_FIRM which is the 3DS' firmware for running DS/DSi games, as well as the Virtual Consoles. I can't personally tell if they're using bilinear or nearest neighbor, but I think it's either bilinear or a combination like hizzlekizzle was stating above. Someone should definitely check me on that though because I don't seem to have that good of an eye for it on their VC's for some reason, but it does look relatively blurry. Perhaps that should be what we ultimately implement for this behavior since it's the closest thing we can get to a pixel perfect fit considering the 3DS' resolution.

Tell me if you guys have other thoughts, though, mine is simply that there isn't any "pixel perfect fit" solution because of the resolution of the 3DS' screen compared to other cnosoles. Then again I only sort of know what I'm talking about (if that) haha so there's likely (hopefully) someone out there to prove me wrong on this.

EDIT: Looks like this is where this feature request should probably be implemented: https://github.com/libretro/RetroArch/issues/2846

[saphtea]

Has anyone else done any further research on this issue?

I think this might be a common issue potentially having to do with how the hardware calculates things. I reported this issue similarly for the Snes9x port by Bubble2k16 here: https://github.com/bubble2k16/snes9x_3ds/issues/38.

They think it has to do with some inaccuracies in the 3D hardware when dealing with floating point numbers, so I'm curious if it has to do with some weird behavior of the PICA2000.

[danieljg]

I recently tested a commercially released 3ds game that very obviously has this same issue. The game is 'bit dungeon plus' and it's so horribly scaled I'm surprised it was released...

Anyways, maybe this will shine some light, as it appears some official software also has the same problem.

[gingerbeardman]

I'm relieved I'm not the only person that this drives crazy

[rsn8887]

All nearest neighbor non-integer scaling looks like this. One solution is to use integer nearest neighbor pre-scale plus bilinear final non-integer scale.

My sharp-bilinear-simple shader does exactly that: https://github.com/rsn8887/Sharp-Bilinear-Shaders

[gingerbeardman]

@rsn8887 thanks for the insight into this, but sadly we can't use shaders on Nintendo 3DS.

[saphtea]

@rsn8887 @gingerbeardman Yes we can use shaders on the 3DS! There are a collection of shaders that are usable on Retroarch's 3DS port although some more intensive ones can cause slowdowns. I think there may be some specific work you have to do to get a normal Retroarch shader to work on the 3DS (as far as I know, I actually have no idea), but it is possible.

EDIT: For the record too I would love to see RSN's shader make an appearance on Retroarch's 3DS port. It results in a really, really clean look and seems like it doesn't have a lot of overhead. Thank you so much for commenting on this thread with it RSN! Hoping to see it come to the 3DS soon :3.

[gingerbeardman]

Oh! Great news @Saphiresurf. I wonder if the @rsn8887's will work?

[saphtea]

Hmmmm, I'm not sure, I don't know quite how shaders work in 3DS' Retroarch at the moment. I don't think it supports GLSL shaders but it does have a lot of shaders built in with the .filt extension and I don't know what else those would be written in unless they're CPU shaders (which I think they may be). Any ideas @rsn8887?

[gingerbeardman]

This could be useful:

3DS shader assembler and disassembler https://github.com/neobrain/nihstro

[rsn8887]

The most portable solution would be if Retroarch could just allow for a software-side integer pre-scale. A simple software pre-scaling before smoothing and final-scaling to fullscreen is applied.

It shouldn't even take much CPU time and it would work on all platforms. The Mame emulator has had this option for over a decade, and so do all the Madmab emulators on classic Xbox.

I honestly don't understand why this simple option is missing in Retroarch. Maybe the existing codebase makes it hard to implement without a large refactoring, but I am just guessing.

[rsn8887]

But the resolution of the 3DS screens is so low anyways that a 2x integer prescale doesn’t even fit any game.

So just turn on smoothing(bilinear) and it should be fine.

[danieljg]

@rsn8887 Regarding the suggestion 'just turn on smoothing (bilinear) and it should be fine', the problem is that bilinear looks pretty horrible, oversmoothed, as you're well aware.

In general, it would be nice to be able to get nearest neighbor working as it should on the n3ds, and your previous suggestion of using a precalc table (probably calculated at configuration time) is probably a good way to go for this case. This would be particularly useful for CPS2 games which have a 384x240 frame which should be presented in 4:3. In the current implementation of RA on n3ds, setting these games to a 320x240 frame results in a messy image that doesn't just drop a column, but has artifacts and looks generaly bad.

A better method would be to render 1:1 and use the pica200 of the 3ds to do the scaling by first blowing it up to 4x, 5x or so, and then reduce to the desired size. Looking at your sharp-bilinear-filters repo, it seems like you're onto something just like that. Kudos.

[rsn8887]

Somehow I just don't understand the problem anymore now that I have looked up the 3DS screen resolution and realized how low it is.

Bilinear filtering is only over-smoothed if the resolution is high enough to support at least 2x pre-scaling. Otherwise, the blurriness you see is just the minimum smoothing required to remove pixel distortion. If you don't like that remaining blurriness, I think there is just no way around it.

Even if the sharp-bilinear-simple shader could be compiled, it would be useless here, because the screen resolution is only 320x240. The sharp-bilinear-simple shader only makes sense if your screen resolution is at least 2x larger than the game resolution, at least in one direction. Due to the low screen resolution, the pre-scale factor the shader will calculate internally will just be 1x. So the result with the shader will be identical to just turning on "smoothing" in Retroarch. Unless you are trying to play games with resolution of less than 160 pixels in the horizontal or less than 120 pixels in the vertical. For such lo-res games, the sharp-bilinear-simple shader would indeed make an improvement.

Regarding the games you mentioned that have a game resolution that is larger than the screen resolution, I never looked into the best scaling for that. It would be great to use the dedicated 3DS hardware to do that. I personally would not like to play hi-res games on a lo-res screen, because pixel information will inevitably be lost.

[gingerbeardman]

The problem is in the OP, check out the first image. Instead of horizontal lines, there are lines with "noise". Here they are again @rsn8887

With Integer Scale Off and Bilinear Off:

With Integer Scale On and Bilinear Off:

[hizzlekizzle]

If you can do shaders at all, you can try setting a pass of the stock shader to 2x scale with nearest filtering and then another pass with "don't care" scale and linear filtering.

[andres-asm]

nope, no shaders on 3ds

[saphtea]

So, since integer scaling 2x -> linear scaling down isn't really going to make a difference due to small screen resolution and you just can't really do proper no-loss, no-artifact nearest neighbor without it being an integer scale, I think the artifact is more of a feature than it is a bug in this case haha.

When we have everything else off it tries to do nearest neighbor scaling at a non-integer value it seems; even though it's not pretty to us, it is very much so doing it's job lmao.

So where do we go from here for more visually appealing or at least less blurry games in retroarch on the 3DS? Well, the current built in filters are a huge save for this, it's unfortunate because it's not perfect, you're not seeing individual pixels with these filters, but depending on the game and console I've come to prefer them to the bluriness. Scale2x I've found works especially well for GB/C games. There is an issue when it comes to the SNES emulators though is that even on a New 3DS the Scale2x filter causes a lot of slow down. Is there any chance that there's performance gains that can be had still with filters with some potential refactoring? Or would the next step be to implement GPU shaders on the 3DS?

Also side note, I'll probably close this issue in a second if y'all agree since it seems like the distortion is caused by proper nearest neighbor scaling, just at a non-integer scale (which will naturally result in artifacts, distortions, and the likes). I just want to start some discussion really quickly to find out where we should try to go from here if anywhere at all so that a proper feature request or something can be made for that. Thank you all so much for your time and input on this so far, it's helped us get another step closer to a better Retroarch experience on the 3DS ^_^.

[danieljg]

When we have everything else off it tries to do nearest neighbor scaling at a non-integer value it seems; even though it's not pretty to us, it is very much so doing it's job lmao.

It's not 'doing it's job' properly at all. Algorithm for nearest neighbor isn't rocket science, those artifacts are clearly not supposed to happen. The psp release does it properly, for example.

[danieljg]

This is very clearly a bug, perhaps a hardware bug on the pica200 side of things as it's been shown to happen in at least one commercial release for the system. In any case, allow me to clarify.

Consider a game for the NES with an output resolution of 256 horizontal by 240 vertical pixels. If one goes for a 'correct' aspect ratio (whatever that means) of 4:3, then the frame must be somewhere around 320 horizontal by 240 vertical.

In this case, it's extremely simple to see that, in a properly-implemented nearest-neighbor filter, one out of every 4 columns of pixels must be doubled, and that's all there is to it. This produces a odd, uneven look, but that's expected.

What is observed in the 3ds release is not that which I've described in the paragraph above, but rather that one out of every five column has this glitchy look where each pixel of the column will be either doubling the correct pixel, or doubling the pixel on the wrong side. The location of the glitchy pixels along the line appear to be random, but they're constant in time.