[Ambient Occlusion] slight artefacts ?

[Boulotaur2024]

I noticed the AO looked somewhat "blocky", not detailed enough as if it was upscaled from a lower resolution (it's not dramatically bad but subtle of course -no offense). So I tried to have a look at the normal map and the problem does arise when normals are created from the depth map :

AO : http://abload.de/img/screenshot_2014-08-17cbuw2.png Normal map : http://abload.de/img/screenshot_2014-08-17zvufv.png

Here is another example in Mafia 2 (using hardware depth but the issue is the same) : ZBuffer : http://abload.de/img/screenshot_2014-08-16jhu7o.png Normal map : http://abload.de/img/screenshot_2014-08-16yzuu6.png

See those dots/discontinued lines around the edge ? I thought at first it was because of a wrong FOV value but after tweaking it to no avail I started investigating a bit more and my guess -its just a guess god I suck at maths- is that it may come from the ddx/ddy operations. Could it be that they create approximations or even errors ? This post seems to explain why (SuperCow's reply) : http://forum.beyond3d.com/showthread.php?t=37614. Yet it seems like ithe right way to do it ! http://www.gamedev.net/topic/512881-normal-from-depth/#entry4340175

So I'm kinda confused as to what to do. I know it happens with HBAO.fx as well. Basically whenever ddx/ddy operations are involved. Think I'll ask for help on Gamedev if we aren't able to find a solution. It's a bit complicated for me anyway :D

EDIT : of course downsampling attenuates the "issue" but it's quite visible at native res

[Boulotaur2024]

I grabbed some code from the ENB forum to generate normals (without ddx/ddy) and it seems to give a cleaner result (this is the same exact frame and the same exact settings) :

original code (ddx/ddy) : http://abload.de/img/screenshot_2014-08-18djf2o.png new code : http://abload.de/img/screenshot_2014-08-18cxe5a.png

Full pastebin : http://pastebin.com/hJ57bFTZ It doesn't have that "aliasing" anymore at all. That said I get a borked, sort of "reverted" AO using this : http://abload.de/img/screenshot_2014-08-18b1egy.png

God I wish I knew what I was doing :)

EDIT : Ah : "If you are doing it after converting a z-buffer depth into a world position, then you will run into continuity problems along edges, simply because of the severe change in Z." http://www.gamedev.net/topic/550356-reconstruct-normal-from-position/#entry4542425

[PeterTh]

That "new code" normal buffer makes little sense to me. Shouldn't the normal of the fish in the foreground and the building wall in the background be almost the same?

Edit: are you sure you aren't storing depth in one component there? Blue perhaps?

[Boulotaur2024]

Yeah these normals may not be very normal lol, hence the borked AO. But my point was that I don't get discontinued/dotty lines anymore using the "new code".

I think I'll ask the question on Gamedev I just registered there because my head is exploding right now.

BTW : Do you have this issue on Dark Souls 2 as well ? If you output the normals only ? Cause I could be completely wrong in my initial assumption that ddx/ddy are the issue

[PeterTh]

I'm not sure I understand your question. The AO I use doesn't use normals, only depth. Also, you wouldn't need to reconstruct normals in DS2, the game renders them to a buffer for you.

[Boulotaur2024]

Correct me if I'm wrong but the VSSAO2 shader does calculate the normals :

    float2 d2 = readDepth(IN.UVCoord);
    if(d2.y < 0.1) return float4(1.0,1.0,1.0,1.0);
    float depth = d2.x;
    float3 pos = getPosition(IN.UVCoord, depth);
    float3 dx = ddx(pos);
    float3 dy = ddy(pos);
    float3 norm = normalize(cross(dx,dy));
    norm.y *= -1;

What happens if you run this http://pastebin.com/SZzKS1vn with DS2 ? I know I get the dotty edges like in the second screenshot in this thread. But I could be alone having that

[PeterTh]

Ah, you're right, I was thinking of the initial SSAO shader back in dsfix. I guess I should try to directly use the normal map from the game.

[Boulotaur2024]

Yeah. If it's anything like the one in Deus Ex for instance http://abload.de/img/testozs01.png then you should get a much higher quality AO. Smooth and curvy hehe. I just learned how better it was to use the vertex normals for AO instead of flat surfaces. Makes sense after all...

As for the ddx/ddy thing well maybe I'm crazy but I read that it could behave differently "depending on the implementation" http://http.developer.nvidia.com/Cg/ddx.html

So maybe it doesn't behave the same on AMD compared to NVIDIA. We'll see... you can close if you want

[PeterTh]

I'll keep it open so that I remember it. Have you tried the GenericDepthPlugin and does everything work for you there? Do you have some profile for a game where AO actually works well?

[Boulotaur2024]

Yeah the integration has been flawless but I think it still needs quite a bit more work to look decent.

I think I solved the dotty lines issues by adapting some code I found to generate normals without ddx/ddy. But now I'd like to get rid of the polygonal look (took a shot with exaggerated settings to highlight the issue. I know this can be toned down -but not eliminated- by tweaking nearZ/farZ). Like I said yesterday I think I'd need to apply SSAO not on surface normals but on vertex normals for this to look much better. I know ENB Boris switched from the former to the latter and he said he'd never go back -understandably so. That said I have no idea how to do that yet : )

[PeterTh]

The issue there isn't so much the "polygonal look", but rather that there is no AO at all. At least I can't see anything in either of those pictures which looks like AO.

[Boulotaur2024]

Yeah lol I'm still struggling a lot with how to read depth properly and the formula behind it. This should be better : http://abload.de/img/screenshot_2014-08-19m9ssw.png

I ported the SSAO shader from here. Reminds me a bit of Crysis AO. 1 pass. Fast : http://pastebin.com/m4RxDt6M

[PeterTh]

That looks a lot more like AO already, good job!

By the way, I love how your GeDoSaTo folder is still Alpha 4 ;)

[Boulotaur2024]

Yeah ! you can call me an early adopter : ) Been messing around with HBAO. Best ssao so far but ridiculously heavy. Not worth it considering the mediocre result :

On/off : http://screenshotcomparison.com/comparison/88479 AO output : http://abload.de/img/screenshot_2014-08-20qnunm.png

Maybe my nzear/farZ are off. It's really difficult to find the right balance and I got sick of alt-tabbing after a while lol

Stumbled on that the other day. Pretty neat : http://fr.slideshare.net/ozlael/mssao-presentation

[PeterTh]

I implemented and tweaked the AO approach from here: http://theorangeduck.com/page/pure-depth-ssao I generally like it, but I need to do something about artifacts for surfaces nearly parallel to the camera plane. And about noise. While also improving performance.

Anyway, I get very nicely detailed results from it: Old: http://abload.de/img/dump004_ssao_md2_buffbdsm6.png New: http://abload.de/img/dump004_ssao_md2_bufffbslr.png

[Boulotaur2024]

NIce !

Is this DS2 ? You're using GenericDepth or DS2 normal map ? That does look a lot better. I like how well defined it is. Best ssao I've seen so far. HBAO doesn't count as it kills fps when downsampling

You might be interested by Martinsh's quick 2x2 blur to smooth the noise (efficient but almost unnoticeable. Worth a try tho) : http://pastebin.com/KM5mQ6DQ

Here's how it looks on Mafia 2 : http://abload.de/img/screenshot_2014-08-22lcp6v.jpg Bit noisy yeah, I get some artifacst as well

EDIT : On/off on Mafia 2 (orangeduck SSAO) : http://screenshotcomparison.com/comparison/88643 http://pastebin.com/4Rw5zLDV

[Boulotaur2024]

I've been trying to make the Crysis AO work for days but something is very wrong I still haven't figured it out. Probably because of the depth reading method again... Here is the shader from Ethatron ported to GeDo : http://pastebin.com/ADvute7w

Would be interested to see if you get it to work with DS2 if you have time

[PeterTh]

I'm closing this now since the initial issue is resolved.