MSAA Artifacts

[Gimlao]

Lines visible between some background elements are visible when enabling anti-aliasing

https://dl.dropboxusercontent.com/u/12308005/Jeux/Fez/screens/MSAAx2.png https://dl.dropboxusercontent.com/u/12308005/Jeux/Fez/screens/MSAAx32.png

[Gimlao]

Random Idea : I'm wondering if replacing MSAA by FXAA would fix that (and other MSAA issues). :3

By the way, it would be nice if the Anti-Aliasing list could include FXAA and MSAA.

[flibitijibibo]

I'll have to look at FXAA. It'd be easy enough to shove this in just before SwapBuffers, and maybe it'll fix a lot of the bleeding that's going on in-game. (But man, MSAA would be nice to keep...)

[flibitijibibo]

After mashing my head against this and working with the FXAA shader...

https://gist.github.com/flibitijibibo/85fc0112ec3a1cff08da

... I've decided to just use MSAA only for first-person mode. The game doesn't look that much better with it on otherwise (in a lot of cases it just looks worse), so it's really just first-person that benefits from this.

In the next build you get MSAA shouldn't show up in 2D scenes.

[renaudbedard]

Chiming in with our potential options...

• Option A : Provide MSAA game-wide. This is still not ideal image quality wise, because edges that should be straight become softer, and there are art-related artifacts that requires a significant amount of work to fix. Other artifacts appear when blitting 0x-MSAA things like text onto the MSAA back-buffer (though we may have a way of fixing this). Biggest advantage is that cutscenes, rotations, first-person view, light shafts, DOT and any rotating geometry gets nice and clean edges.

Some artifact examples :

• Option B : Provide MSAA only in cases where it doesn't cause visual issues. This means that in regular gameplay it would not be enabled, we would only toggle it on in first-person view and possibly cutscenes. This may feel like a bug to players.
• Option C : Provide FXAA game-wide. We already decided against it because it looks like crap.

With FXAA : Without :

• Option D : Don't expose any anti-aliasing in the game menus, but support it and let people who want to tinker in the Settings file enable it at their own risk. This means it would be unsupported and we don't spend any more time making it work right.

And ideas on the best way to go?

[Gimlao]

I would say that letting people to use FXAA, if they want, could not be a bad thing. I still prefer FXAA rather than MSAA, but only for 3D scenes. 2D scenes will always be ruined by any filter.

By the way, blurry things are never pleasant for the eyes. I never liked to use any filter in a 2D console emulator, it will always ruin the original pixel art.

I also consider that 3D pixel art exist. =p And since FEZ is a game with trixels... :3

[renaudbedard]

Spoke with Gimlao in twitter DM for a while... He and I are leaning towards leaving MSAA and FXAA as options, but not in the menu, in the Settings file so they are basically unsupported.

The game doesn't really need MSAA to look correct, and it causes issues that are probably not going to get resolved in time. But tearing it out completely is a bummer, and enabling it selectively is weird, so leaving it as a hack is the lesser evil... I think.

Same idea for FXAA : I hate how it looks but some people might enjoy it, and at this point it's cheap to support. I still don't want to expose it in menus because it might lure people into thinking that it's an optimal way to play the game (turn all the settings to Ultra!!)... it's not. Leaving it as an unsupported flag leaves it open for hacking around.

Thoughts?

[flibitijibibo]

What I could probably do is keep the option in the menu, but only make it visible with --msaa-option, so it's still technically unsupported, but those who want to mess with it can make use of the in-game functionality without having to restart every time they want to experiment with different modes. The config var would always work, but would need restarts without the in-game option.

I'm somewhat hesitant to have FXAA paths because what few places it worked in were only better when the max MSAA support was 8x, and the hardware that doesn't support at least 16x probably can't handle the fragment shader that I put together (it's 253 instructions including multiple branches with an additional preshader, which might be a new record for what I've seen). FXAA would also have to be first-person only, because text was nearly unreadable when FXAA was enabled (remember, linear filter is required!).

At this point I'm only really considering whether or not to apply the MSAA effect globally or just in first-person mode. Since it's unsupported I could just do it globally, but I dunno if the various visual problems are worth the not-nearly-as-many benefits that you get outside of first-person.

[renaudbedard]

MSAA : I'd push for globally, especially because the 32-cube and 64-cube cutscenes look really good with MSAA and it's be a shame for nobody to ever see it... ^_^ I'm still planning to try and fix the text blitting with MSAA to use only one sample, and perhaps look at the animated sprite texture clamping... so that'd be two less artifacts.

Hiding behind a CLI flag sounds good to me, better than having to edit the Settings file.

FXAA : I really don't care for it. Let's just not do it. Sorry Gimlao!

[flibitijibibo]

I've made the following changes that should show up in the next update:

MSAA, when enabled, will apply to the whole game, not just first-person. MSAA's in-game option is NOT visible by default. The in-game option will show up with either --msaa-option or setting multiSampleOption true in Settings.

There will still be issues when out of first-person, but I'm putting it down as WONTFIX since it's not officially supported at this point.

[renaudbedard]

Sounds good! I'll leave this thread open for the two MSAA fixes I want to do (mentioned above).

[renaudbedard]

My first fix (disable multisampling when drawing non-multisampled things) is blocked by https://github.com/flibitijibibo/FNA/issues/333 right now.

[flibitijibibo]

This is one of those times where having an XNA4 version really comes in handy... if we could show the XNA and FNA versions side-by-side to a vendor with the GL_MULTISAMPLE call being ignored, we could at least get an application profile that makes it do the right thing, but without that we'd have to hope that the RasterizerStates are just correct and that they'll just take our word for it.

[renaudbedard]

I... don't have an XNA version of the game. I could try building one but I'm not very excited about spending time doing that. I understand what you're saying though. I have emails for both an AMD contact and an nVidia contact, I can try hitting them up to see how receptive they are. Do we know for sure that AMD drivers have the same issue?

[renaudbedard]

I'll close this as "oh well, we tried".