search

RPCS3 / rpcs3

PlayStation 3 emulator and debugger
https://rpcs3.net/
GNU General Public License v2.0
15.41k stars 1.92k forks source link

Ratchet & Clank: Tools of Destruction/Resistance: Fall of Man instanced mesh system broken #10754

Closed JimScript closed 5 months ago

JimScript commented 3 years ago

Quick summary

In some early Insomniac games, only one foliage instance can be visible at a time

Details

In Ratchet & Clank: Tools of Destruction, New minor foliage will replace rather than supplement other older foliage. Foliage fades in as normal: screenshot-2021_08_23_15_09_26 screenshot-2021_08_23_15_09_37

And then when a new foliage loads in it completely replaces the old one: screenshot-2021_08_23_15_09_48

Now this would be a minor nitpick, if Resistance: Fall of Man (another insomniac game) didn't use the system more extensively and for a gameplay purpose like hiding mines.

Here is an RSX Capture: BCUS98127_20210823151612_capture.zip

And here is the system:

JimScript commented 2 years ago

Well I might as well show an example of Resistance: Fall of Man: BCUS98107_screenshot_2021_11_20_04_05_48 BCUS98107_screenshot_2021_11_20_04_05_52 BCUS98107_screenshot_2021_11_20_04_05_58

RSX Capture: BCUS98107_20211120040735_capture.zip

JimScript commented 2 years ago

Sorry if this is confusing, here is a video of it in action:

https://user-images.githubusercontent.com/66685584/142732140-f40f2eef-92c8-493a-adfe-36ea8a87b6e0.mp4

JimScript commented 2 years ago

Here is a visual aid on the differences between ps3 and rpcs3 foliage sorting logic in Ratchet and Clank Future: Tools of Destruction. PS3: FoliageDemoPS3 RPCS3: FoliageDemoRPCS3

Sorry I couldn't get a photo or video of a real ps3, I don't own a capture card or something like that. But there is a surprising amount of foliage missing in these games because of this error, more so than I originally thought.

JimScript commented 2 years ago

Ok, so I figured my phone picture was good enough to compare so... PS3: 0212221204 1

RPCS3: BCUS98127_screenshot_2022_02_12_12_08_42

And with this I see that some of the foliage is in clusters that are never seen even in the culling order that RPCS3 produces for some reason. I wonder what causes it to only show one mesh at a time, but 2-3 meshes in Resistance: Fall of Man?

JimScript commented 2 years ago

Along with the previously stated sorting issue, it appears that some foliage is rendering only as an outline. This example is in Quest for Booty, but it has much the same rendering as ToD does, so it is applicable to both games: NPUA80145_screenshot_2022_02_15_14_14_41 RSX Capture: NPUA80145_20220215141422_capture.zip

JimScript commented 2 years ago

OK, so I have been playing around with renderdoc to see what the problem is and this is what I found. The missing foliage is well and truly not there, no draw calls except for one solitary plant in this foliage system, which leads me to believe this is a cpu problem. I would guess this is an spu problem as a task like this seems too minor for the ppu. What I think what is suppose to happen is this spu function is to go around and update the foliage to different levels of opacity as the camera gets closer, but in RPCS3 the function stays at the first foliage it can find until it has to make a unique mesh fully opaque wherein it goes to the new foliage and dumps the old one.

As for the outline shader, I may not know much about these things, but I don't think a single minor mesh is suppose to draw over a thousand times in a single frame. A third of the frame render time is dedicated to that one plant, and to add to that the first several hundred draws just draw the same few pixels on the screen. Although that could be just how the game does it, but that seems like a horrible optimization.

Anyway that is just some speculation on my part.

kd-11 commented 2 years ago

Needs testing on AMD/INTEL. NVIDIA is notorious for having rounding issues that can cause strange visuals like this.

Shadowsplace commented 2 years ago

I tested it on two different machines in Tools of Destruction and got the same result.

https://user-images.githubusercontent.com/100016968/154794078-005533c0-7fb9-48c2-82e4-9464bf43c466.mp4

https://user-images.githubusercontent.com/100016968/154793778-d76f0045-4d25-4daf-b5e6-c7f8aad328a8.mp4

Both using Vulkan but OpenGL doesn't seem to change anything (on Nvidia at least).

JimScript commented 2 years ago

I think there are two separate issues in this issue, the foliage sorting and the outline shader. I think what kd-11 is talking about is the shader issue. What you have tested is the sorting issue. If you want an example of the shader problem in action, go to hoolifar island in QFB. If you don't have QFB, Planet Mukow's foliage has the same problem, so you can check it there.

JimScript commented 2 years ago

Hey @Shadowsplace, can you check the foliage in the imperial fight festival with your AMD machine? It is different than the ones on cobalia and I was wondering if @kd-11 is right about this particular part of the issue is vendor related. And sorry I couldn't do it, I don't own any AMD gpus. Here is an example of the problem on Nvidia hardware: BCUS98127_screenshot_2022_02_27_14_49_44 If your AMD machine still makes the foliage here an outline, then it isn't vendor related.

Shadowsplace commented 2 years ago

Sorry about the delay. Same specs (RPCS3 v0.0.21-13333-aafd74f9 Alpha | HEAD build)

It appears the issue isn't vendor related.

JimScript commented 2 years ago

Thank you for testing this, Shadowsplace, with this we now know both issues are not vendor related. I still think that the sorting part is a spu(or ppu) problem, but I don't know what could cause this particular shader to render as an outline.

JimScript commented 2 years ago

I was looking at an old resistance video when I noticed that the amount of foliage missing is worse than Tools of Destruction's. Here is a screenshot of RPCS3 in roughly the same location as 0:28 in the Somerset(40p) map: BCUS98107_screenshot_2022_05_11_15_06_20

homembarata commented 2 years ago

Wish I had seen this thread before...

It's not just grass, but small stuff like small rocks are also missing. Here's a later level in Resistance: rpcs3 exe Screenshot 2021 07 10 - 02 08 59 27

In the original hardware there are a bunch of small debris on the floor (not exactly the same screen but the same area): Resistance_ Fall Of Man _ PS3 _ Full Game Playthrough Walkthrough _ No Commentary 3-57-4 screenshot

My guess is that this is an instanced mesh system, which is common for this kind of small objects and foliage. And the emulator is only rendering one instance at a time. Just throwing some ideas here.

JimScript commented 2 years ago

It probably is that since I made a Renderdoc of this issue and didn't find any of the missing meshes anywhere in it. But it also seems to be different than the fade in/out as that works correctly with the one mesh that does render. Also certain multiplayer maps can run more than one at a time for some reason.

isJuhn commented 2 years ago

I believe ratchet games culls lots of stuff in case of low fps before the rendering stage at all. I've checked a lot of ToD renderdocs and if it runs too slow a lot of stuff is not even in the renderdoc. But at the same time the foliage is never there no matter the fps so it might be some stage before the render stage that removes them, and not some rsx culling.

JimScript commented 2 years ago

I have thought a bit about the possibility of this being related to a performance cull, and now I think that this culling system might not be receiving a proper(or any) performance report. That would explain why there is generally only one mesh rendered at a time, because to the instanced mesh system the PS3 is getting absolutely slammed, so it renders the bare minimum amount of meshes it can(only one). That would also explain why Grimsby(40p) in CTF can render more meshes at once, because it is getting a lighter report. Now this is all guesswork, but it is the only thing that I can think of that would cause this.

Darkhost1999 commented 2 years ago

I have to host the video somewhere else because GitHub will say the video is corrupt or too big. Oblivion (looks better upscaled and if I load all of the environment first.) Trees lose leaves as you approach them. https://youtu.be/rBs_uqE2V1k RPCS3.log

OpenGL Comparison https://youtu.be/8nowDtWDMJY RPCS3.log

I know this is a different game. Spooky. But the issues look the same. I mean let's compare Oblivion only loses leaves not entire plants. All 3 games lose foliage at similar distances to each other. The other games plants are on the ground. Oblivion's leaves are off the ground. If you want a separate issue ticket just lmk.

I can test on my Intel uhd 620 igpu laptop later for a comparison there? Would that be helpful or because igpus are not officially supported would it be meaningless?

kd-11 commented 2 years ago

This one never got triaged. I've added it to backlog.

JimScript commented 2 years ago

I have to host the video somewhere else because GitHub will say the video is corrupt or too big. Oblivion (looks better upscaled and if I load all of the environment first.) Trees lose leaves as you approach them. https://youtu.be/rBs_uqE2V1k RPCS3.log

OpenGL Comparison https://youtu.be/8nowDtWDMJY RPCS3.log

I know this is a different game. Spooky. But the issues look the same. I mean let's compare Oblivion only loses leaves not entire plants. All 3 games lose foliage at similar distances to each other. The other games plants are on the ground. Oblivion's leaves are off the ground. If you want a separate issue ticket just lmk.

I can test on my Intel uhd 620 igpu laptop later for a comparison there? Would that be helpful or because igpus are not officially supported would it be meaningless?

I'll admit, I don't see the similarity. In TOD & R:FOM they both can only show one or few pieces of foliage at a time with them popping around as you get closer or farther, this seems more gradual(except for the trees suddenly popping up in the background in the Vulkan example). I wouldn't even say it was the outlined plants part of this issue, as that remains consistent at any distance with any renderer. But maybe the underlining renderer could be used differently?

Darkhost1999 commented 2 years ago

I'll admit, I don't see the similarity

You never see similarity to ANYTHING I say so IDK OGL did same thing as vulkan tree pop in I just needed to load the water so there actually is no difference between renderers

Darkhost1999 commented 2 years ago

I definitely see as you described this RPCS3: FoliageDemoRPCS3

JimScript commented 2 years ago

I'll admit, I don't see the similarity

You never see similarity to ANYTHING I say so IDK OGL did same thing as vulkan tree pop in I just needed to load the water so there actually is no difference between renderers

I'm sorry about that. But issues with Oblivion were fixed with the PC uptime problem I posted, so maybe it is similar and would be fixed with this.

JimScript commented 1 year ago

I think the reason why some of the ground plants look hollowed out and spindly is that that the alpha is limited to around 127 and anything past that is all zeroed out for that shader, making the bulk of it not render. This is the alpha of the QFB plant edited to demonstrate: QFBPlantBrokenAlpha And the original to compare: QFBPlantOriginalAlpha

JimScript commented 1 year ago

I've found something interesting about the hollowed out shader: BCUS98127_screenshot_2023_09_24_22_10_18 It doesn't block shadows, even though it does on a real PS3. It makes sense since this is a transparent material, but what if it can block shadows on PS3 because it's filled with an opaque material, like it fades from transparent into opaqueness. And what if it's hollowed doing around a thousand draw calls because it's caught in a loop trying to get the opaque version but it's just getting the transparent version over and over again until the emulator gives up? I guess I'm making a lot of assumptions here.

JimScript commented 1 year ago

Turns out it does draw the opaque part first in another color pass, but it only does so on a very slim threshold. That must have been the few pixels I saw being drawn first before, I could only really see it now when the LOD bias offset is set to 10 since the threshold is so tight. Opaque Shader Drawn: ToDPlantOpaque

Translucent Shader and Shadow Drawn: ToDPlantTranslucent

Still don't know why both the opaque and translucent parts draw hundreds of times each frame though.

kd-11 commented 6 months ago

Test https://github.com/RPCS3/rpcs3/pull/15471. It's not directly aimed at this issue (I haven't deep-dived this one yet) but the cause could be related.

rogeriodomingos commented 6 months ago

I can still replicate the issue in Resistance: Fall of Man as of version 0.0.31-16351-ea891829.

https://github.com/RPCS3/rpcs3/assets/10940771/f53dd15f-a78f-4576-844c-32cb0238a52a

JimScript commented 6 months ago

You know, with Resistance: Fall of Man showing multiple clusters rather than a single plant, could that in fact be a different issue? I mean this issue turned out to be different from the one that affected Resistance 2 and ACIT on NVIDIA, who's to say this isn't a different issue either? Still, this is only speculation, I won't make another issue until we know for sure.

rogeriodomingos commented 6 months ago

It might be the same issue, but it has improved. Here is a screenshot with version 0.0.30-15956: Screenshot 2024-04-18 135822

And version 0.0.30-16096 with the same behavior as the previous video I posted: image

https://github.com/RPCS3/rpcs3/pull/15471 doesn't seem to have made any difference in this case. I'll try to narrow down the commit that made the improvement.

rogeriodomingos commented 6 months ago

It was actually this pull request that did it: https://github.com/RPCS3/rpcs3/pull/15065

I see that you also reported an improvement in Tools of Destruction so it might be related.

Summing things up: https://github.com/RPCS3/rpcs3/pull/15065 improved the issue but https://github.com/RPCS3/rpcs3/pull/15471 had no effect.

kd-11 commented 6 months ago

This one is weird. I'll have to investigate assuming it's multiple issues. Hopefully it's not related to depth precision since that can only be correctly resolved with software rendering.

kd-11 commented 6 months ago

Ok, nothing too special here. It's what I expected. PS3 doesn't have dedicated instancing hardware, but it can do instancing through like a million different methods by maniupulating the FIFO puller. We just didn't have a pattern match for the registers this game is using to implement it. Fix incoming.

kd-11 commented 6 months ago

https://github.com/RPCS3/rpcs3/pull/15483 implements the feature, though in a very suboptimal way right now. A lot more work is needed to get this ready for merge.