EnableOverscan = True messes up native resolution modes

[dankcushions]

[Video-General]
ScreenWidth = 1920
ScreenHeight = 1080

[Video-GLideN64]
# Enable resulted image crop by Overscan.
EnableOverscan = False
# PAL mode. Left bound of Overscan
OverscanPalLeft = 0
# PAL mode. Right bound of Overscan
OverscanPalRight = 0
# PAL mode. Top bound of Overscan
OverscanPalTop = 0
# PAL mode. Bottom bound of Overscan
OverscanPalBottom = 0
# NTSC mode. Left bound of Overscan
OverscanNtscLeft = 0
# NTSC mode. Right bound of Overscan
OverscanNtscRight = 0
# NTSC mode. Top bound of Overscan
OverscanNtscTop = 0
# NTSC mode. Bottom bound of Overscan
OverscanNtscBottom = 0

UseNativeResolutionFactor = 1
EnableHybridFilter = False

And then change:

[Video-GLideN64]
# Enable resulted image crop by Overscan.
EnableOverscan = True

(most obvious in the signatures and top line of text)

It seems that enabling overscan is messing up the calculation of the native resolution - perhaps it's not being excluded/included at the right point?

[gonetz]

There is no major bug here. The problem is the follow:

• The game uses RDP color buffer of size 440x330 to render that screen.
• This RDP buffer is mapped on video output image of size 440x240 in Video Interface. That is the output TV image has less vertical lines than the original one in RDP.
• When user uses native-res without overscan, the 440x330 image is mapped to the main buffer of window resolution, say 1024x768
• When user uses native-res with overscan, the 440x330 image is first mapped to 440x240 aux buffer. Then, part of that aux buffer rendered to our main 1024x768 buffer. If overscan trim left/right/top/bottom settings are all zero, the whole aux buffer is rendered to the main buffer. This is your case.

So, the source of image degradation in overscan mode is render of 440x330 image to 440x240 one. angrylion's plugin works as GLideN64 in overscan mode, that is it maps 440x330 RDP image to 440x240 TV output. As the result, GLideN64 without overscan looks better than angrylion's even in native-res mode. But downscale from 440x330 to 440x240 makes the result look worse. However, the hybrid filter helps to improve the situation. Moreover, GLideN64 with the hybrid filter still looks slightly better than angrylion's :

[dankcushions]

Thanks for the example!

I guess I'm a little confused what overscan means in this context. My understanding of overscan in other emulators is that CRT TVs don't often show the user the full image. eg, a typical 320x240 NES might really appear as 224x192 or something around that, because some is hidden behind the bezel, or not shown on the tube at all:

so EnableOversan = True would imply to me that I would see the full 440x330 of Goldeneye, and EnableOversan = False would imply to me that the edges are going to be cropped by some amount. Say, 400x300. it wouldn't imply to me anything different about the scaling algorithm used. It seems like this option should be renamed?

As to what the scaling is doing:

I know the N64 real hardware is weird and I believe everything ends up to 640x480 before it arrives on the TV (which I guess the nativeres feature + overscan off bypasses and I'm thankful for that!) but I don't know where these 440x240 or 1024x768 buffers come from? in this old thread about angrylion, they mention that goldeneye gets scaled from 440x330 to 640x480 which introduces some scaling artefacts: http://forum.pj64-emu.com/showthread.php?p=55354#post55354, but that is different to what's happening here.

Also, my understanding of the hybrid filter was that it was a way of reducing scaling artefacts when going from rendered resolution (low res) to display resolution (high res). I don't think it should be applied during any internal downscaling to intermediary buffers, even if the effect is pleasing.

[gonetz]

so EnableOversan = True would imply to me that I would see the full 440x330 of Goldeneye, and EnableOversan = False would imply to me that the edges are going to be cropped by some amount. Say, 400x300.

conversely, EnableOversan = True means that the edges are going to be cropped by some amount. That's why I recommend to use large enough native-res factor with Overscan. For example, if your output resolution is 1024x768, use native-res factor 4 to get rendered image larger than your output resolution. In that case, image quality will not degrade after the crop.

Note, that overscan applied not to RDP image, but to output of Video Interface, exactly as on the real hardware. In this GE example, Video Interface maps 440x330 RDP image into 440x240 TV image. Overscan crops that 440x240.

but I don't know where these 440x240 or 1024x768 buffers come from?

TV image has either 240 lines or 480 interlaced lines. Most of N64 games use TV image with 240 lines, which can be shown on TV screen without interlacing, that is without flickering. Number of pixels per line can be arbitrary in range from 320 to 640. Usually that number is equal to width of RDP image, but it is not always so. In our case Video Interface (VI) maps 440 pixels of RDP image to 440 pixels of TV image. 440 pixels are scaled onto 640 dots on TV scanline, it is TV hardware work. The plugin emulates VI, so it maps maps 440x330 RDP image into 440x240 one. The 440x240 image is then cropped by overscan and rendered to output buffer, which is displayed on user's monitor in user-selected resolution, say 1024x768.

Also, my understanding of the hybrid filter was that it was a way of reducing scaling artefacts when going from rendered resolution (low res) to display resolution (high res).

Yes, the plugin uses hybrid filter to upscale rendered image to display resolution, if necessary.

I don't think it should be applied during any internal downscaling to intermediary buffers, even if the effect is pleasing.

Yes, downscale uses plain nearest neighbor filtering. However, I don't agree that hybrid filter should not be used for downscaling even if it works better. As we see here, nearest neighbor filtering works poorly if image is low-res.

[dankcushions]

conversely, EnableOversan = True means that the edges are going to be cropped by some amount.

this is explicit in the .cfg annotation, so my fault :)

# Enable resulted image crop by Overscan.
EnableOverscan = True

That's why I recommend to use large enough native-res factor with Overscan. For example, if your output resolution is 1024x768, use native-res factor 4 to get rendered image larger than your output resolution. In that case, image quality will not degrade after the crop.

in my examples i'm using 1080p, native res 1, but with overscan = true it's very degraded?

In our case Video Interface (VI) maps 440 pixels of RDP image to 440 pixels of TV image. 440 pixels are scaled onto 640 dots on TV scanline, it is TV hardware work. The plugin emulates VI, so it maps maps 440x330 RDP image into 440x240 one. The 440x240 image is then cropped by overscan and rendered to output buffer, which is displayed on user's monitor in user-selected resolution, say 1024x768.

So I'm not sure how EnableOverscan = true affects this process, because with overscan = false it's: 1) start 440x330 2) to 440x240 3) to display resolution

...i would expect to see similar scaling artefacts as overscan = true: 1) start 440x330 2) to 440x240 3) to ~400x200 (i don't know how many pixels overscan=true crops). 4) to display resolution

the latter case would be worse, but the difference in quality should not so huge.

it seems to me like overscan = false bypasses the 440x330 to 440x240 step completely, as if i do a quick pixel count on my first example, it seems like each native pixel is 3-4 pixels high, in the final 1080p image, which would make the native resolution somewhere between 270-360 pixels high, rather than 240. ie, i think it's 440x330, and has missed this 440x240 step.

i also don't see any cropping take place when overscan=true. in your examples, should't the "TWYCROSS BOARD" text be closer to the top of the image in the second example, compared to the first, if cropping has happened?

(sorry for all the questions!)

[oddMLan]

@gonetz

Yes, downscale uses plain nearest neighbor filtering. However, I don't agree that hybrid filter should not be used for downscaling even if it works better. As we see here, nearest neighbor filtering works poorly if image is low-res.

Yeah NN isn't going to work well for downscaling. My intention was to do the following:

• Bilinear filtering always for downscaling.
• For upscaling Hybrid Filter if enabled, else use NN.

Because the Hybrid Filter is going to be wasted processing when downscaling (is not going look any different that plain bilinear). Hybrid Filter is only useful when upscaling. Maybe I didn't represent this correctly in the code.

[gonetz]

@dankcushions

So I'm not sure how EnableOverscan = true affects this process, because with overscan = false it's:

1. start 440x330
2. to 440x240
3. to display resolution

No. with overscan = false it's:

1. start 440x330
2. to display resolution When overscan is disabled, RDP image is mapped to output image directly.

it seems to me like overscan = false bypasses the 440x330 to 440x240 step completely,

Yes.

i also don't see any cropping take place when overscan=true

It is because I set all "overscan.top/bottom/left/right" to zero, so nothing is actually cropped, but the step to 440x240 is performed because overscan is enabled. For most of games overscan with zero crop should look the same as with no overscan. It is very rare case when RDP image is larger than output TV image.

in your examples, should't the "TWYCROSS BOARD" text be closer to the top of the image in the second example, compared to the first, if cropping has happened?

You need to compare the whole image. I made some manual crop of screen shots just to illustrate image quality. However, some disproportions are possible when overscan is enabled, because the image is scaled twice.

@oddMLan

My intention was to do the following:

• Bilinear filtering always for downscaling.
• For upscaling Hybrid Filter if enabled, else use NN.

I was requested to use NN for downscaling because Bilinear makes image blurry. Hybrid Filter is used only for upscale indeed.

[dankcushions]

the step to 440x240 is performed because overscan is enabled

i presume the step to convert to 240 only happens when the real hardware is in progressive (240 lines) mode, and note interlaced (480 lines) mode? are we sure that goldeneye was not upscaling to 480i in these scenes? i found this discussion:

http://www.shootersforever.com/forums_message_boards/viewtopic.php?p=54691&sid=4eba4817147db52d505c0c5e782a98d8#54691

GoldenEye's high-res mode, as Dragonsbrethen stated, is 440x330. This, I presume (and if I'm wrong here, I'm wrong about everything after this point), is scaled up to fit the N64's 640x480i high-res mode. Compared to GoldenEye's regular 320x240 mode, you're getting 1.375 times the vertical and horizontal picture information. Unless my math's off, this means the image is just a bit over 27% larger.

this makes more sense to me, as a downscale from 330 to 240 lines would introduce some quite horrible artefacts that should be visible in angrylion/real hardware, unless some kind of bilinear filter is applied during this downscale? it would also seem silly to develop the game to render 330 line 2d menus if it has to be downscaled later.

I was requested to use NN for downscaling because Bilinear makes image blurry. Hybrid Filter is used only for upscale indeed.

this doesn't seem to be the case when overscan = true:

1. start 440x330
2. to 440x240
3. to display resolution (1080p)

if hybrid filter is only applied in step 4, it shouldn't make an obvious difference to the image quality at 1080p, because of the high pixel density - you would have to zoom in. however we can easily see the difference between these two 1080p screenshots:

hybrid filter off

hybrid filter on

i am convinced that hybrid filter is being applied to step 2, a downscale. if it was nearest neighbour those images should look effectively the same until you zoomed right in.

[gonetz]

are we sure that goldeneye was not upscaling to 480i in these scenes?

Yes. Hi-res mode can be enabled later in game settings. I checked, interlaced mode is not used for this screen.

this makes more sense to me, as a downscale from 330 to 240 lines would introduce some quite horrible artefacts that should be visible in angrylion/real hardware, unless some kind of bilinear filter is applied during this downscale?

Image quality degradation with angrylion is clearly visible on my comparison picture. The first picture with GLideN64 in native mode without overscan looks way better than angrylion's (AL RDP).

it would also seem silly to develop the game to render 330 line 2d menus if it has to be downscaled later.

Yes, it is unusual way to render 2D.

if hybrid filter is only applied in step 4, it shouldn't make an obvious difference to the image quality at 1080p, because of the high pixel density - you would have to zoom in.

I noticed that NN works bad for low-res images, so hybrid filter is now used for downscale native-res, see 2f5280c8ac So, 440x330 => to 440x240 is done with hybrid filter if it is enabled. The result is close to angrylion's, may be because it also uses bilinear for downscale.

[dankcushions]

Image quality degradation with angrylion is clearly visible on my comparison picture. The first picture with GLideN64 in native mode without overscan looks way better than angrylion's (AL RDP).

yes you're right, but i guess that first example is kind of a 'fake' if you're saying that 'overscan = true' turns on the emulation of the VI downscaling to 240 lines, which is what real hardware or angrylion seems to do. however, comparing the second example (overscan + hybridfilter) and the angrylion look very similar, so i can agree that the degradation seems equal.

I noticed that NN works bad for low-res images, so hybrid filter is now used for downscale native-res, see 2f5280c So, 440x330 => to 440x240 is done with hybrid filter if it is enabled. The result is close to angrylion's, may be because it also uses bilinear for downscale.

for raspberry pi and other low bandwidth devices, we can't often afford to have hybrid filter on as it impacts performance, and the benefits when applying it to a typical 1080p upscale is slight (although it's a cool effect!). would it be possible to instead apply 'free' bilinear filtering to these kind of downscale operations, if hybrid filter is disabled? and still leave the final upscale to display resolution as NN.

[gonetz]

would it be possible to instead apply 'free' bilinear filtering to these kind of downscale operations, if hybrid filter is disabled?

Yes, it is possible. Note, that we are talking about very rare situation: downscale of native res image.

[oddMLan]

would it be possible to instead apply 'free' bilinear filtering to these kind of downscale operations, if hybrid filter is disabled?

Yes, it is possible. Note, that we are talking about very rare situation: downscale of native res image.

I was requested to use NN for downscaling because Bilinear makes image blurry.

I noticed that NN works bad for low-res images, so hybrid filter is now used for downscale native-res, see 2f5280c

The result is close to angrylion's, may be because it also uses bilinear for downscale.

I don't believe that bilinear would make the picture blurry when downscaling. Bilinear only makes images blurry when upscaling. Hybrid filter (in theory) should produce the same result as plain bilinear when downscaling, due to the way it works internally. It would have to resample all the pixels. Hybrid filter applies resampling selectively when upscaling.

Could you please compare downscaled native res GoldenEye picture (overscan on) between bilinear and hybrid filter? And just to be safe, compare downscaled high res picture (2x native) on a 640x480 window the same way, with a different, more colorful game like Mario Tennis.

[gonetz]

Mario Tennis hardly has situation when native res image is downscaled. It is very rare case. Hi-res image downscale (2x native and above) currently uses NN. I may compare NN-vs-bilinear in downscale, later.

[dankcushions]

Fixed via https://github.com/gonetz/GLideN64/pull/2313: