Inaccurate dithering matrices

[TorutheRedFox]

The matrices for dithering aren't accurate to hardware. The N64 doesn't use typical dithering matrices like most PC games and the PS1 for example.

GLideN64:

AngryLion/hardware:

[TorutheRedFox]

Although I do think that the ordered-like matrices should be an option for those who prefer them.

[gizmo98]

Maybe this happens because we use a real 4x4 bayer matrix if bayer dithering is selected. And as you can see the result looks like dithering with a bayer matrix 😉. Angrylion‘s bayer matrix/pattern didn‘t look like a real bayer matrix so i did not use it.

[TorutheRedFox]

it should be an option for accuracy

[gizmo98]

Yeah. Programming manual tells me a 4x4 bayer matrix is used but angrylion‘s matrix does not look like a bayer matrix. Which documentation is right?(screenshot of real hardware?). It is easy to add another 4x4 matrix. So we can check if angrylion‘s matrix produces the same result.

[TorutheRedFox]

the AngryLion matrix looks to be accurate to the real hardware to me (different game but same pattern is visible)

[TorutheRedFox]

I have no idea why the N64 uses such a wonky dither matrix. Maybe to them it just looks better on common gradients?

[TorutheRedFox]

it's probably optimised to work better with the VI filter

[Jj0YzL5nvJ]

Personally I do not distinguish dithering in CRTs... https://www.reddit.com/r/n64/comments/j8wy5g/picked_up_a_jvc_flat_crt_for_free_ive_never_seen/

P.D: I prefer the blue noise =P P.D 2: I want black and white output support, that's my childhood.

[TorutheRedFox]

on hardware, the dithering is blurred by the VI filter, making it create a flat colour or a smooth gradient

[TorutheRedFox]

unless you get rid of it with hardware mods that is

[gizmo98]

For accurate representation we would need a blur VI filter which mixes colors and recreates original blurry look😉.

[TorutheRedFox]

I mean it's just FXAA with depth based edge finding and a slight horizontal blur

[gizmo98]

A slight horizontal blur would remove dithering fragments. Angrylion bayer matrix looks worse even with blur applied (((x-1) + x + (x+1))/3).

[oddMLan]

I wonder why they bothered with the blurring step, it looks worse and dithering is not even that noticeable in a CRT.

[TorutheRedFox]

idk either

[gizmo98]

@oddMLan I read somewhere n64 dither filter should not blur 2D elements too much. I modified blur filter like color = texel(x-1)0.25 +texel(x)0.5 + texel(x+1)*0.25. Dithering disappears or looks smoother but 2D elements do not look to much blurred.

[gizmo98]

Here are some real ingame blur screenshots. I replaced fxaa shader with the above mentioned blur shader. It does not look so bad. I like the idea to enable blur and aa if a game requests these vi features. The right bayer matrix could be added as well.

Without Blur: With Blur:

Without Blur: With Blur:

Without Blur: With Blur:

Without Blur: With Blur:

Without Blur: With Blur:

[oddMLan]

Looks really close to hardware now but it should still be made optional thru the UI imo (and possibly only enable if the dithering is visible, otherwise it's moot). Are you also giving the option for "accurate" N64 Bayer as well?

[TorutheRedFox]

the FXAA should also use a mask so it doesn't affect 2D elements of the image, and be less noisy (N64's AA is smooth)

and maybe, if possible, but I doubt it, do the weird polygon antialiasing that the N64 does (which does real MSAA-like antialiasing, but only in places where polygons directly touch), and make the FXAA use depth differences to detect edges, rather than colors

[gizmo98]

Come on. You cannot do antialiasing with interpolation or signal filtering (blur is in the end a vertical low pass). ;-) FXAA needs coverage calculation and this is out of my league.

I will try to add blur alongside the current fxaa and add angrylion's bayer matrix.

[gizmo98]

@ToruTheRedFox You can test two branches. https://github.com/gizmo98/GLideN64/tree/blur2 has N64 bayer matrix and enables dither filter if dither filter vi register is set. There is no option atm to disable dither filter. https://github.com/gizmo98/GLideN64/tree/n64bayer just has N64 bayer matrix.

[TorutheRedFox]

I can't build the source myself, it just errors out in many different ways no matter what I try

[Jj0YzL5nvJ]

I can't build the source myself, it just errors out in many different ways no matter what I try

Try with:

git clone https://github.com/gizmo98/GLideN64.git
cd GLideN64
git checkout blur2
curl -L -o ..\2358.patch https://github.com/gonetz/GLideN64/pull/2358.patch
git apply ..\2358.patch
projects\msvc\GLideN64_builder.cmd --x86 --noqt --nom64p

The script will require some external CLI tools like 7z.exe, git.exe and curl.exe in PATH (rundll32 sysdm.cpl,EditEnvironmentVariables).

If it fails at compile time after that, your Visual Studio installation is incomplete or corrupted. Visual Studio installer does not work OOTB by clicking next, next ...install. In fact, if you do that, you will get an incomplete and mostly useless installation, guaranteed.

[TorutheRedFox]

nah my VS installation is fine as it builds everything else fine

it just can't get the git revision and a few static libraries for some reason

[Jj0YzL5nvJ]

nah my VS installation is fine as it builds everything else fine

So does that mean you can build Project64? BTW, I recently fixed a bug related to the revision script for GLideN64... #2354

[TorutheRedFox]

I haven't tried building Project64 because I didn't have the need to

but I have built AngryLion before

[Jj0YzL5nvJ]

Project64 may fail to generate Version.h in similar fashion to GLideN64. Angrylion-Plus is unexpectedly tolerant to revision failures...

Wrong revision: https://i.pinimg.com/originals/05/dc/2f/05dc2fa43266c99d47c2d63e8a43d213.png Expected revision: https://i.imgur.com/7oP8Ux6.png

[gizmo98]

@ToruTheRedFox The right pattern should be in master now. I tested it with dither test software http://mattpierce.info/n64-dither.

My dither filter also does not look so bad. I added logic which does not blur text, and borders/edges:

Without this logic text and 2D elements look bad:

[TorutheRedFox]

I still get the old pattern for some reason (I use "Render N64 framebuffer to output")

[gizmo98]

NativresFactor = 1 and RDramDithering is off? Dithering pattern is certainly not right if NativeResFactor != 1.

[TorutheRedFox]

my configuration is this

[gizmo98]

Please enable „Dithering pattern on output image“ and „5Bit quantization“. Disable RDRam dithering. These settings are most accurate because dithering is only used if a application requests ist. RDRam Dithering option is only usefull for framebuffer effects if higher resolutions are used.

[TorutheRedFox]

I'm still not getting correct results

[gizmo98]

@ToruTheRedFox I thought about it. I also used a real magic square matrix instead of angrylions matrix. With angrylions matrix it looks like this:

With blur:

Angrylion's magic square matrix can be found in this branch: https://github.com/gizmo98/GLideN64/tree/blur3

[oddMLan]

My dither filter also does not look so bad. I added logic which does not blur text, and borders/edges:

Is it possible to add this logic to the FXAA filter?

[gizmo98]

In my blur2 (always on) and blur3 (ini option, default off) branches it is a post filter which runs after fxaa.

[gizmo98]

Can one of you verify it looks better or accurate now?

[TorutheRedFox]

from the screenshots you posted, it does, but as I can't compile it myself due to missing dependencies that I don't have the drive space for, I can't test it myself either

[gonetz]

I can't compile it myself due to missing dependencies that I don't have the drive space for, I can't test it myself either

This fix is in the master now, so you may download necessary build from AppVeyor: https://ci.appveyor.com/project/gonetz/gliden64/branch/master

[TorutheRedFox]

except it isn't

I checked

[TorutheRedFox]

dither is the same as it has been

[TorutheRedFox]

well the matrix is there, but it's still not drawing correctly for me (yet another Intel issue?)

[TorutheRedFox]

I don't think the dither texture is being applied with the correct UVs to the output (framebuffer dither works fine)

[gizmo98]

Mario64 seems to use magic square pattern. I have angrylion‘s ms pattern in my branches. The result can be seen above. I will do a PR if someone tells me the pattern is right now.

@gonetz appveyor can only be used for upstream/master? I use a linux buildscript for testing. No option for W64/Project64 Users.

[oddMLan]

AppVeyor allows building PRs and other branches, but gonetz explicitly disabled the feature for whatever reason

[gonetz]

gonetz explicitly disabled the feature for whatever reason

and now can't remember why.

[avdland]

wasnt that because of the website? so that you will download latest master and not some PR?

[gonetz]

wasnt that because of the website? so that you will download latest master and not some PR?

No. The link in README.md points to master branch builds.

[gonetz]

Should not this issue be closed? 93ddc890e, ec02cc7

[TorutheRedFox]

oh right