Weird Band in the Scanlines

[dwynator]

Hi, Release 20200922. of the TurboGrafx16 core has a strange problem. On HDMI1080p With vscale_mode on 0.5x, I got Banding on the top of the screen. Only visible with the Scanline-Filter which I always use. Seen best with a stronger Scanline Effect (I Tried all). When I switch the "Overscan" to "Visible", the Scanlines appear Spot-on! As if the Overscan on "Non-Visible" not scaling correctly, hence the Banding Effect! I Tried to reproduce the same problem with vscale_mode on 0.25x, and I got the Banding with Both Overscan to "Visible" and set to "Non-Visible". The Later setting did not resolve it as with 0.5x setting but made the scanlines look garbled. Setting the vscale_mode to Integer Scaling Works perfectly, but makes for a smaller screen area.

I added a Picture, I hope you can see because I play on a Projector. Thanks for the awsome work!! Hope you'll look into this. Goodluck, T.

[sorgelig]

i don't remember any changes might affecting it. Try to switch to RGB palette (as it was in previous releases).

[M-Walrus]

Does this issue only happen in the new core released on 9-22-20? Not in the previous releases? Those types of banding issues can happen with any core if you are not using integer scaling (vscale_mode=1) with the scanline filters. Switching the viewable area between overscan visible/not visible, can make these issues more apparent, as you seem to have discovered here. To prevent ever seeing issues like this, it's best to keep scaling at perfect integer (vscale_mode=1), when using scanlines.

[birdybro]

Unless you are using integer scaling and unless your output resolution supports 1:1 scaling the resolution (like 1440p perfectly scales 240p source up 6x), then the scanlines will always be occasionally bloated. People call this "shimmering" when you move, I prefer just saying uneven scanlines and uneven pixels behind it.

You are using 1080p output, so some scanlines will necessarily be bigger no matter what. It's math. 1080/224= 4.821428. So 4x224=896. 1080-896=184. Now, if you are going full screen, they have to distribute 184 pixels vertically as extra, to fit everything. If you are using 0.5x, then it still gives you borders and reduces this by a bit, but it's still uneven and still distributing across vertically extra pixels, still making them uneven.

If you turn on integer scaling, you will have a 92 pixel border on top, and another 92 pixel border on bottom in 224p on a 1080p display. :)

[dwynator]

Does this issue only happen in the new core released on 9-22-20? Not in the previous releases? Those types of banding issues can happen with any core if you are not using integer scaling (vscale_mode=1) with the scanline filters. Switching the viewable area between overscan visible/not visible, can make these issues more apparent, as you seem to have discovered here. To prevent ever seeing issues like this, it's best to keep scaling at perfect integer (vscale_mode=1), when using scanlines.

I Tested the last 2 Cores. 1080p is The Resolution. When Overscan=Hidden, you see the banding. (Most visible with the Darker settings of the Scanline-Range for example: Scanlines (Softer)"Catmull-Rom_Scanlines_100 or Scanlines (Sharper) "Normal_Scanlines_100") It's as if the Overscan=Hidden "setting" doesn't scale optimal? (Hence the bands when placing Scanline filter over it?? Or the Filter itself needs different settings? I'm just guessing here of why this occurs with the TurboGrafx Core and not with NES, SNES, SEGA etc.) With the vscale_mode=0.5x "setting" in contrast to the Overscan:Visible "setting", which makes the "Visible" Image size the same as if vscale_mode:Integer Scaling is used and both have perfect results. Again, other Cores I tested: NES, SNES, Sega Genesis/MD. Work Perfectly with the vscale_mode: 0.5x setting + 1080p (Which I like because it makes a bigger image on 1080p compared to Integer) P.S Thanks, I get that the easy solution would be Just indeed do integer scaling but indeed makes it smaller.. But when in vscale_mode= 0.5x, Is it not the amount of Overscan that's been hidden that causes this? Forgot to tell (Picture doesn't show) the Bands are also in the lower part of the screen. Thanks for the quick help on the issue!

[dwynator]

Unless you are using integer scaling and unless your output resolution supports 1:1 scaling the resolution (like 1440p perfectly scales 240p source up 6x), then the scanlines will always be occasionally bloated. People call this "shimmering" when you move, I prefer just saying uneven scanlines and uneven pixels behind it.

You are using 1080p output, so some scanlines will necessarily be bigger no matter what. It's math. 1080/224= 4.821428. So 4x224=896. 1080-896=184. Now, if you are going full screen, they have to distribute 184 pixels vertically as extra, to fit everything. If you are using 0.5x, then it still gives you borders and reduces this by a bit, but it's still uneven and still distributing across vertically extra pixels, still making them uneven.

If you turn on integer scaling, you will have a 92 pixel border on top, and another 92 pixel border on bottom in 224p on a 1080p display. :)

Thanks for clarifying. Yes that "shimmering" I get what you mean, I get it when just "Scaling to Fit Screen".. It's just strange that it only happens with Turbografx Core on the Exact same 1080p, vscale_mode=0.5x setting. For Example if I change the "Hide Overscan" on the NES Core either On or Off, all stays well and perfectly fine with the Scanlines (No Banding at all), which make me think it sits in the "Overscan" with the TGFX16 Core?? Because with it set visible (Border), TGFX16 does display the Scanlines fine. Can any of you reproduce this problem? Thanks.

[dshadoff]

TG16 core with overscan area is capable of displaying more vertical lines than NES. This affects the multiple that fits on the screen.

If vscale mode was integer, you wouldn't see banding (but you would feel that there is too much unused display area).

[Kitrinx]

Scanlines will only be fully consistent with full integer scaling because that's how math works. This has nothing to do with the core at all.

[dwynator]

I understood the TGFX16 Core was rewritten? Does it have the PhaseAcc mode scaler? The Person (This was 2018) has had The Same Problem with the NES Core Back Then.
See this thread on AtariForum Here

[dwynator]

TG16 core with overscan area is capable of displaying more vertical lines than NES. This affects the multiple that fits on the screen.

If vscale mode was integer, you wouldn't see banding (but you would feel that there is too much unused display area).

Aha?! So if the TGFX16 Core on 1080p 0.5x mode, has good Scanlines with Overscan=Visible(="More Vertical Lines than NES" ) but The Banding occurs with Overscan=Hidden.. Then a re-Written Filter could or may resolve the Banding? Or is it something that needs a look in the TGFX16 Core code? This PhaseAcc mode scaler? has come up in the Link of my prev. Reply. Thanks people!

[dshadoff]

No, it's like @Kitrinx said - it's math. Switch to integers and all will be well. Or use a different resolution display like 720 or 1440.

TG16 is pretty much continuously variable in terms of what a game can choose to display and what is displayed in overscan area. Very different from Nintendo and their enforced programming standards. Any game can create a situation where a different number of display lines are active, thus altering the multiplier.

[Kitrinx]

Again, banding has absolutely nothing to do with the core. It's how scanline filters work. Depending on a core's resolution the un-evenness may vary in screen location, but the ONLY way to get even scanlines is to use integer scaling. Not 0.25, not 0.5, but ACTUAL integer scaling. It's not a bug, it's just how division works.

[birdybro]

This is just math. When you don't have 1:1 pixel scaling in the ratio to what you are stretching, you have excess pixels that the stuff displayed on your screen has to compensate for. I already explained this to you.

[sorgelig]

Ok. This is not an issue. Some filters like scanlines require integer scaling to eliminate the artifacts. So use integer scaling. Closing this as it's not an issue. Welcome to MiSTer forum to discuss if you want learn more about it.