Mesen issue? Shader issue? Neither? Border effect not applied with crop

[klepp0906]

First, I have to apologize. This veers dangerously close to support due to my ignorance on the matter.

I spent a lot of time setting up RA over the years, with many platforms all with custom aspect ratios to fit my overlays. In time ive come to care more about an accurate image and decided to go through things with a comb one more time. Huge undertaking.

Starting with mesen-s i noticed its defaulting to crop-none. Now im sketchy on overscan in general and the different implementations with some crop on = showing overscan, some crop on = removing it.

I would assume based on wording with mesen crop-none, means its not cropping anything, but seems its the opposite right?

First question is general. It appears snes is supposed to be 256×224 yes? Does this mean with crop-none thats exactly what we're getting? Thus with crop 8 vertical and horizontal we are getting 272x240? 16 results in 288x256?

or is crop 8 a 4 pixel addition to each side and crop 16 an 8? (probably this right?)

question still stands, does that mean instead of displaying 256x224 would that mean with crop 8 we are getting 264x232 on screen? With crop none vs crop 8, it seems to zoom, but on the top and the bottom it seems to be actually adding rows of pixels as well.

I ask this for a few reasons. As noted i want to get things right and not touch this again. I need to fill in gaps in my understanding so im no longer entirely ignorant as to how this works. I realize overscan is the pixels that generally fall behind a tv's borders and are unseen, but does this mean that the full x16 setting is actually the 256x224 and setting it to none would just be "zooming" the image?

the more practical reason i ask (and the title of my post is due to this issue im finding when playing with a particular shader relative to overscan).

I'm upgrading from barebones geom to royale during this process and royale applies a really nice darkening/softening effect along the edge of the rendered display so as to simulate the transition on crt tv edges.

here is the shader with crop set to none

(ignore the horizontal stretch, im using my custom AR as i wanted to keep the image in my old overlay to illustrate the contrast on the edge)

notice how the effect is applied to the sides, but not the top?

now here it is with crop set to 8 on all sides

see how the effect is now applied to all sides?

It seems like horizontal overscan is literally just zooming, while vertical is zooming and chopping off a part of the screen?

So my first priority is to get the effect applied to all sides (which i obviously can do by setting overscan to 8) but is this the true full image? (with overscan now "hidden" by the effect rather than a tv's physical border?) or do i want 16? and either way why the discrepancy between the settings functioning on the sides vs the top/bottom with the sides not changing rendered dimensions but the top seemingly chopping rows going from none to 8 (but not 8 to 16)? Irrespective of all of the aforementioned, shouldnt the shader be applying to the top and bottom edges regardless of overscan setting?

I apologize for the book, i was messing with this all night and figured id better come to the source as whats out there on the net is seems subjective and fragmented.

[SourMesen]

The overscan settings cut off pixels from the screen. The SNES can output up to 256x239, and it has a mode to cut this down to 256x224. 256x224 is what you get if you set vertical overscan to 8px, which should be all you need?

[klepp0906]

ok so first and foremost, why does horizontal crop settings changing not seem to change the aspect ratio of the display area, whereas vertical lops off the top and bottom then?

horizontal does seem to zoom it in and out, it moves ui elements closer to the edges of the display - but it doesnt change the dimension of the display area like going from none to 8x on vertical does. (16x does no further growing or shrinking, just seems to "zoom" once again)

Is this all due to the fact its vertical resolution operates as a range, whereas the horizontal is a rigid 256?

Does this mean it truly is just zooming on the horizontal? Should i be leaving horizontal set to none? or still set it to 8?

also with the vertical you say it operates 224-239 and setting it at 8 will make it 224? so setting it to 16 will give you 239? so what does none give you? 216 i guess? That being outside the "range" you mentioned - is that why the shader is missing/not adding the effect to the tops and bottoms in that instance?

i appreciate you taking the time to help me understand this a bit better. could have just as easily been a developer from another core, but i started with yours lol.

[SourMesen]

Cropping does not affect the aspect ratio, no. The aspect ratio is a pixel aspect ratio (e.g NTSC has 8:7 pixels), so removing or adding pixels to the picture does not alter their ratio.

There is no difference in behavior between horizontal and vertical cropping - if retroarch reacts differently, then that most likely is just retroarch. I know it doesn't tend to resize the window when I change these settings, etc, so that may be why you thinking the options do not have the same effect.

Like I said in my previous reply, if you want to cut off the black bars on the screen, set the vertical overscan to 8px (this will crop pixels on both the top and bottom to get the picture down to 224px), and keep horizontal overscan to 0. The value for the setting corresponds to the number of pixels you want to cut off, not the amount of extra pixels you want to show.

[klepp0906]

ok final question as im understanding more of this much more now.

the horizontal overscan is not adding or removing black bars no matter the setting. It simply changes the way the image displays. hence with 0, 8, 16 the shader border that began this saga is always displayed.

the vertical overscan perhaps does not change the "aspect" ratio as you noted, it just appears to due to black bars lopping off some of the image. This only happens at 0. Would also indicate that the shader IS actually applying the effect, its just black over black, meeting black and thus cannot be seen.

why does vertical add black bars whereas horizontal does not? Is this a fundamental snes thing tied to the fact that the resolution is always output at 256xVariable?

youve helped me understand that what i want for a proper image as it would have been displayed on a tv is the 8px on the vertical and the core provided aspect ratio.

I'm just trying to understand the horizontal portion in part in case you return telling me its a game by game thing, and in part for emulation in general going forward.

i do sincerely appreciate your time. I'll close this after i hear back on this last tidbit.

[SourMesen]

Again, the overscan settings cut out pixels on the screen. The snes outputs 256pixels wide, if you set it to 8px horizontal, you are cutting off 8 pixels on both sides of the screen, and end up with a 240px wide picture.

The black bars at the top/bottom are because the SNES can output either 239 or 224 lines, based on a flag games can alter. Most games output 224 lines, which is why there are black bars, because the core always outputs 239 lines.

The horizontal overscan setting is not particularly useful for the SNES - it only exists because it was copied over from Mesen's code, and NES games do tend to display garbage on the edges of the screen (because of HW limitations and/or bad programming), and the overscan settings are there to hide them.

For the SNES, set the vertical overscan to 8px and call it a day, nothing else is useful or should matter much.

[klepp0906]

ok, semantics i suppose. obviously youre correct, im just interpreting it as zoom or stretch when it is removing pixels - its just maintaining the aspect ratio specified.

i was thinking of it in a more literal sense, largely due to the fact that the top and bottom actually DO seem to do that (lop off screen area) when in fact its there, i just cant see it.

I'm ocd if that much hasnt been made obvious by now.

again, your time is appreciated a tremendous amount. thanks for holding my hand on something that should have probably been more easily understood.