[Feature request] Pass GSDX Depth information to external program

[SirFancyBacon]

Hello Team, This is my first feature request; second "issue" ever posted to github, please forgive my ignorance and poor formatting.. :)

Would it be possible to pass/expose depth information (assuming the PS2 has a functioning depth buffer) onto a program like Reshade (i am speaking specifically for Reshade in this request)?

The advantages of this change would be: 1.) Ability to filter out "HUD" elements from the 3D geometry when improving lighting effects. 2.) Ability to use Reshades more advanced libraries; such as SSAO, DoF, and others i likely haven't thought of.

I have fallen in love with making custom shader settings using the built in GSDXFX shader file; as the PS2 is by far my favorite system the ability to "remaster" these classic titles is something i would love to have.

I made a separate post instead of posting this in Gregory's #2310 as this isn't an "optimization"

[mirh]

@masterotaku maybe you know something about this

[masterotaku]

With 3Dmigoto you can modify shaders and use depth information to do different things. I use it to convert shaders to 3D, ignore certain depth values to not stereoize unwanted things (like HUD), and similar stuff. But I almost never used Reshade and have no idea about how it works. You'd better ask the 3Dmigoto or Reshade creators to have more information. I just use 3Dmigoto to modify existing effects.

[SirFancyBacon]

@masterotaku You have much more experience with 3Dmigoto than i, and likely would have a better idea on who to contact; Would you mind doing the leg work on that side?

I've started with this post on the Reshade side of things: [https://reshade.me/forum/general-discussion/3981-ogl-pcx2-support-depth-information] We can use this bug report to collaborate on the topic; provided you have the interest or the time that is :)

[gregory38]

There is a depth information in the PS2. We emulate it with a depth buffer. However it could be used in akwards ways.

[SirFancyBacon]

Could you elaborate? Awkward like a weird way to do depth information? Or to be used as something other than depth information?

Is there any point as/after the data is calculated and it resembles something half way normal it could be passed along to the render API ?

[masterotaku]

From the games I tried (30-something) only God of War 1 and 2 had weird depth. Most of their geometry is within a very small depth range, definitely not normal. For 3D I made a workaround to expand that area with some drawbacks.

I know three people with at least some experience about Reshade (1000 times more than me), and the first two are the main 3Dmigoto developers: https://github.com/DarkStarSword, https://github.com/bo3b, and https://github.com/Kaldaien .

[masterotaku]

Umm, did that count as summoning someone? Maybe I just had to write @DarkStarSword , @bo3b and @Kaldaien .

[mirh]

Two invocations in a day must be quite of a tiresome call for Andon.

[gregory38]

Most of the times, depth should be close. What I mean is that depth is an integer (16/24/32 bits) so it can be read/written as a rgba color. Besides the buffer is mutable. On gpu you define a buffer size, origin, format when you allocate the buffer (and you can't change it). On GS, those parameters are set for each draw call. So the size of your depth buffer can change in the middle of the rendering. The origin too (the pixel 0,0). Again, it doesn't mean all games uses awful tricks, but some game might do.

[SirFancyBacon]

Well... no luck on the Reshade side of things; my post received 95 views and no replies.. I ask for documentation so perhaps i could look through it and bounce ideas off of everyone.

If only i could choose which effects to layer and when with GSDX.FX then i'd at least have SOME of my desired features

[SirFancyBacon]

Got a response from the devs:

"There is no documentation. But here are the relevant code locations that take care of debug buffer detection:" hooked opengl calls related to fbo setup On FBO Attachement Depth Source Table Depth Source Table#2

I fear i've done all i can; I am a good power user; but can't code to save my life..

[gregory38]

Could you fix your link as I did for the first one ;)

[SirFancyBacon]

Whoops! sorry; very new to Github :) comments corrected :D

[gregory38]

I don't know if it is relevant but potentially depth buffer is unattached of the FBO if there is no depth test.

[gregory38]

@hellbringer616, would you be able to breakpoint in reshade code to ensure code is correctly called. Or break in GSdx depth attachment and step by step into reshade.

[SirFancyBacon]

I... am going to guess no; at least not without guidance. My coding experience is; I once told my web browser to say "Hello World" in Javascript. I once compiled some code and got nothing but errors; on code that functions perfectly fine for everyone else... :)

But, with that out of the way; i don't mind giving things the ole college try! However i can't touch any code until Sunday afternoon (3/11/18) -5GMT; if i'm lucky.

Google tells me a breakpoint is an intentional pause in the code for debug purposes; if you know how i can do that (and how to compile if that is also needed) then i'll get it done.

[mirh]

Also of note https://github.com/crosire/reshade/pull/40

[SirFancyBacon]

Very nice, Looks like it's time to wait for the next Reshade release! hopefully i can figure out whats up with OGL not loading in the meantime (between studying and work; not likely soon..)

[Boulotaur2024]

You might want to check out this thread here : https://reshade.me/forum/general-discussion/5391-release-pcsx2-with-depth-buffer-access

I hacked together a slight variation for Reshade to get access to the z-buffer of Pcsx2 (OpenGL and D3D11) to perform effects like SSAO, DOF, and SSR (written by Reshade developpers, not by me)

It works pretty well but some games (Metal Gear Solid 2, Persona 3/4, Nocturne...) refuse to display some kind of zbuffer (it mostly shows a black screen and only the HUD does have some kind of "depth to it").

I would really love Pcsx2 dev to chime in and tell me... WHY : ) Is it overwritten in the process ? Is it cleared ?

Other than that, I'd say 6 out of 10 games are working fine (in OpenGL). Have fun :]

[gregory38]

When does reshade need depth information. During rendering ? Only at the end for post processing effect? How reshade read the depth buffer ? Texture bound to current framebuffer depth slot ?

[gregory38]

Note: for 24 bits depth you need a 256 factor. And for 16 bits a 256*256 factor.

[Boulotaur2024]

Gregory, hello : ) And sorry for not replying earlier but 1 month ago I had no idea about how to reply to your questions. Now I'm a little bit more educated...

So this is how Reshade reads the depth buffer with Pcsx2, by hooking SwapBuffers() and injecting the following code :

// Cycles through all the FBO and retrieve the FBO the "latest" and most used one (according to number of drawcalls)
// Here for Pcsx2, _depth_source = 3 (Framebuffer 3 is chosen - see screenshot below)
GLInt _depth_source = detect_depth_source();

// Copy back buffer to RBO
glDisable(GL_SCISSOR_TEST);
glDisable(GL_FRAMEBUFFER_SRGB);
glBindFramebuffer(GL_READ_FRAMEBUFFER, 0);
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, _fbo[FBO_BACK]);
glReadBuffer(GL_BACK);
glDrawBuffer(GL_COLOR_ATTACHMENT0);
glBlitFramebuffer(0, 0, _width, _height, 0, 0, _width, _height, GL_COLOR_BUFFER_BIT, GL_NEAREST);

// Copy depth from "detected" FBO (see detect_depth_source()) to depth texture 
glBindFramebuffer(GL_READ_FRAMEBUFFER, _depth_source);
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, _fbo[FBO_BLIT]);
glBlitFramebuffer(0, 0, _width, _height, 0, 0, _width, _height, GL_DEPTH_BUFFER_BIT, GL_NEAREST);

// Render effects (SSAO, FXAA, etc...)  
update_and_render_effects();

// Copy results from RBO to back buffer
glDisable(GL_SCISSOR_TEST);
glDisable(GL_FRAMEBUFFER_SRGB);
glBindFramebuffer(GL_READ_FRAMEBUFFER, _fbo[FBO_BACK]);
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);
glReadBuffer(GL_COLOR_ATTACHMENT0);
glDrawBuffer(GL_BACK);
glBlitFramebuffer(0, 0, _width, _height, 0, 0, _width, _height, GL_COLOR_BUFFER_BIT, GL_NEAREST);

// Flip to screen
SwapBuffers();

_on_drawcall() is called after each call to :

• glDrawElementsBaseVertex
• glDrawArrays

It tries to get the latest currently bound framebuffer and eventually (well I'm taking a small shortcut for clarity) assigns a value for __depthsource.

Thing is we are having a big "delay" issue now between the ssao buffer and... what I guess is the backbuffer (color buffer ?) : the depth buffer is always "too early" and shows frames "from the future" :D

I was talking about it in Reshade forum : https://reshade.me/forum/general-discussion/5391-release-pcsx2-with-depth-buffer-access?start=20#34412

Note : the D3D11 backend shows exactly the same issue, so it's not an OpenGL thing... Note : every emulator is affected by that (pcsx2, redream, gliden64...)

Gonetz (GlideN64 developer) is aware of the issue and we are currently trying to find a solution, for GlideN64 anyway : https://github.com/gonetz/GLideN64/issues/1548#issuecomment-506668328 EDIT : he solved the issue !

I thought you might be interested Gregory, your knowledge could be valuable.

Reshade build for testing : https://mega.nz/#!TU8DFQxY!43yGDASXCnGGH1T-IAJaPJw0cMqRw4ckMGUe3ed7nGU In the OpenGL tab you can check "Force default depth buffer" to read from framebuffer 0 instead of non-zero framebuffer

[gregory38]

I don't understand. It seems the code is reading from framebuffer 0. But IIRC, fb 0 means the window itself (so not the standard fb object). On the window buffers we shall only use the color (aka back buffer). So I don't know what data you get.

[gregory38]

Ok. I read N64 comments. I understand what happen. What we shall do

Thanks for the info. I will describe later what we need to do

[Boulotaur2024]

I don't understand. It seems the code is reading from framebuffer 0. But IIRC, fb 0 means the window itself (so not the standard fb object). On the window buffers we shall only use the color (aka back buffer). So I don't know what data you get.

Oh my bad, I made a test yesterday and I'm pretty sure I could get the depth buffer by reading framebuffer 0, but I must have been mistaken... or because I used a SVN build from May 2019... Because now I get nothing showing (no depth) if I comment _detect_depthsource() (I'm using very latest SVN build -has something changed since then ? ok nevermind)

I will update the above code so that it reflects the correct behaviour

[gregory38]

Not necessarily a bad thing. If we track the depth we can still copy it in window buffer.

Annoying issue is that we need to keep the depth alive. Depth might not exist anymore when presenting the frame

[gonetz]

Depth might not exist anymore when presenting the frame

I solved it in GlideN64 by keeping depth buffer snap shot for each color buffer.

[Boulotaur2024]

Gregory, hello. Any news ? I know we all have lives and busy schedule but... tu es mon seul espoir Unless you can point me out to relevant code that needs modifying and in that case I could make the changes myself

[ghostinthecamera]

Sad to see no update on this :( would have been great to get the depth buffer working in MGS2

[tadanokojin]

Greg's awfully busy these days. Even I don't see him much now.

I understand the enthusiasm for something like this but we are in a freeze right now so new features like this aren't a priority. Hopefully we can get the release out soon and then maybe someone can look into it. If someone else wants to in the meantime, that's alright too.

[SirFancyBacon]

Two things; first happy thanksgiving all! Second... very excited to see movement on this; obviously i am beyond behind on these updates lol

[RedDevilus]

Hmmm ReShade 5 improves depth buffer passing 5.0.1 explicitly mentions a fix for PCSX2 depth buffer detection. Might be worth checking it out.

[Jakey757]

Hmmm ReShade 5 improves depth buffer passing 5.0.1 explicitly mentions a fix for PCSX2 depth buffer detection. Might be worth checking it out.

it does detect something but nothing actually able to be used. i was able to get some menu boxes in games to show up but at the wrong sizes.

heres DW6 briefing screen. you can make out the very front 2d silhouette. this is in vulkan, ill have to check the other renderers.

edit: same thing in dx11

edit#2: the generic depth detection add-on will cause PCSX2 to throw asserts and or freeze the application when toggling some shaders using depth, or when toggling the conservative/large framebuffer flag.

[jacklollz2]

Would love to see depth buffer to be sent, not just for Reshade but for Nvidia filters as well.

[Levan7]

This would be great with vulkan api

[Gunlaus]

@Boulotaur2024 Could you explain to me how you do the scale/offset calculation to get the depth to align? Latest reshade works with pcsx2 but i can't seem to get the alignment perfect