RPCS3 / rpcs3

PlayStation 3 emulator and debugger
https://rpcs3.net/
GNU General Public License v2.0
15.33k stars 1.91k forks source link

Call of Duty: Black Ops II [BLUS31011] Game freezes randomly #13602

Open Darkhost1999 opened 1 year ago

Darkhost1999 commented 1 year ago

https://github.com/RPCS3/rpcs3/issues/13602#issuecomment-2392530127

Old issue
When booting BLUS31011 updated to 1.19 the game crashes at the press start screen if using 100% default configuration. But by changing zcull to approximate or relaxed we can actually play the game. This same fatal error and device lost affects so many of my games that I am glad to have obtained this game and am able to make a replicable debug output log. ``` ·F 0:00:58.200436 {RSX [0x80558bc]} SIG: Thread terminated due to fatal error: Assertion Failed! Vulkan API call failed with unrecoverable error: Device lost (Driver crashed with unspecified error or stopped responding and recovered) (VK_ERROR_DEVICE_LOST) (in file C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\rpcs3\Emu\RSX\VK\VKQueryPool.cpp:45[:4], in function poke_query) (in file C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\rpcs3\Emu\RSX\VK\vkutils\shared.cpp:107[:24], in function die_with_error) ``` ### Attach a log file [Debug.log](https://github.com/RPCS3/rpcs3/files/11130782/RPCS3.log) [Crash.log](https://github.com/RPCS3/rpcs3/files/11130783/RPCS3.log) [Working-Approximate Zcull Accuracy.log](https://github.com/RPCS3/rpcs3/files/11130785/RPCS3.log) ### System configuration Ryzen 5 3600 GTX 1660 Super Windows 11
kd-11 commented 1 year ago

If it's consistent on your system you can install SDK and capture a log with debug output enabled. Also missing steps to reproduce, it's a large game, I don't know if it crashes in the menus or a specific level, etc.

Darkhost1999 commented 1 year ago

If it's consistent on your system you can install SDK and capture a log with debug output enabled.

Debug.log

Also missing steps to reproduce, it's a large game, I don't know if it crashes in the menus or a specific level, etc.

When booting BLUS31011 updated to 1.19 the game crashes at the press start screen if using 100% default configuration.

kd-11 commented 1 year ago

I'm unable to reproduce on RTX 3050. The debug output also has no information about any violation that could cause a crash. There is some log spam about use of barycentrics, but that shouldn't crash and has nothing to do with zcull accuracy. You can easily prove that this is not the culprit by setting shader quality to low.

kd-11 commented 1 year ago

I have an idea - try this build: https://cirrus-ci.com/task/5489746785337344 Let's see if it solves the nvidia silent crash problem.

Darkhost1999 commented 1 year ago

There was a change. The fatal error is now before the intro cutscene rather than after. Debug.log RPCS3.log

kd-11 commented 1 year ago

We just have to wait for nvidia to implement VK_EXT_device_fault then. AMD already ships that extension, basically whenever the driver crashes it should give some more information to us about why it did so. Until then I think it's not very useful to keep fighting the NVIDIA driver.

kd-11 commented 1 year ago

Try with https://github.com/RPCS3/rpcs3/pull/13875

Darkhost1999 commented 1 year ago

This is using https://github.com/RPCS3/rpcs3/pull/13875/commits/bc9633396e35180563215c4341f0374f392b158b image RPCS3.log -Edit add debug output log- RPCS3.log

Shoegzer commented 1 year ago

Getting something similar here, though not quite the same. The game doesn't crash at the start screen on default settings, but rather freezes ingame after a while (between 3-15 minutes), usually in the "zombies" mode. Using default settings except for Approximate ZCull accuracy and Ordered&Atomic RSX FIFO accuracy. Note though, the "device lost" message isn't thrown to the log file. The log below is quite large as it took about 15 minutes of play to trigger the freeze.

OS: Linux Mint 21.1 / kernel 5.15 CPU: Intel i7-10870H GPU: Nvidia GTX 3070 / driver 530.41 Commit tested: 1371bf89e04eedb4879ce97562af4f07f392f45d Log: RPCS3.log.zip

Shoegzer commented 1 year ago

Update: Gave this a closer look. As I mentioned I don't see the VK_ERROR_DEVICE_LOST exception in the log, though I do see CELL_NET_CTL_ERROR_NOT_CONNECTED as @Darkhost1999 has reported:

19386, spu:0]; [soft:665 hard:727]; Switches:[vol:187178 unvol:14150]; Wait:[9.577s, spur:131010]
·E 0:21:02.668006 {PPU[0x1000000] main_thread} SYS: Stat: 'cellNetCtlGetInfo' failed with 0x80130108 : CELL_NET_CTL_ERROR_NOT_CONNECTED [x2012]

This is soon followed by other entries that seem to reflect a problem, for example:

·W 0:21:02.668038 {PPU[0x100000a] Thread (hang detect) [0x0033d844]} PPU: 'sys_timer_usleep' aborted (9.539612s)

Failed attempts to resolve include:

Note that I cannot test OpenGL as rpcs3 hangs during the initial shader build process (though this happens with all titles, not just this one).

Darkhost1999 commented 1 year ago

Issue not occurring on RPCS3 v0.0.29-15463-edf4f7ea Alpha RPCS3.log

Shoegzer commented 1 year ago

Still freezing here unfortunately, after several single-player zombie mode games. Tested against commit edf4f7eacc4d1c87fc1fade32f1662e49e2748ad with new cache and default configuration as well. Please re-open.

·E 0:25:41.404741 {Emulation Join Thread} SIG: Thread [rsx::thread] is too sleepy. Waiting for it 4226396.399µs already!

Log: RPCS3.log.zip

Darkhost1999 commented 1 year ago

Looks like when I tried today. Reinstall your game update files and try again. https://wiki.rpcs3.net/index.php?title=Help:Installing_Game_Updates#Removing_game_updates/game_data

Shoegzer commented 1 year ago

Thanks for reopening. I tried what you suggested but it still froze. Update path was from 1.00 > 1.09 > 1.19, was that yours?

Log: RPCS3.log.zip

Asinin3 commented 1 year ago

Test with https://github.com/RPCS3/rpcs3/pull/14657 (if its merged when you're reading this, just test latest RPCS3 build instead)

Shoegzer commented 1 year ago

Thanks, still freezes after 5-15 minutes after going ingame in testing a build with #14657. Again I do not see VK_ERROR_DEVICE_LOST in the logs but rather the others as I noted above.

New log in case it helps: RPCS3.log.zip

Darkhost1999 commented 1 year ago

Yea freezes for me as well. I don't obtain freezes when zcull accuracy is changed from precise.

Shoegzer commented 1 year ago

Well I can certainly create a new issue for this, as this one's title may be misleading assuming the fatal device error isn't a cause of freezing any more.

With regards to zcull accuracy, I've tested using precise and approximate and freezing occurs with either (though perhaps not as frequent with approximate). Play in zombie survival mode, specifically starting in the bus station, and you should see the same thing?

kd-11 commented 1 week ago

Does the driver still crash @Darkhost1999 ?

Darkhost1999 commented 1 week ago

RPCS3.log.gz On RPCS3 v0.0.33-16999-938306a7 I can not finish the first mission when using precise zcull. My old comment I assume holds true

I don't obtain freezes when Zcull Accuracy is changed from precise.

kd-11 commented 6 days ago

Well, the log doesn't show any RSX crashing though. The only difference between precise and approximate is that precise makes the RSX code run slowly through the same steps. I've updated the title to reflect that this issue is not causing a device lost error anymore.

Shoegzer commented 6 days ago

Freezing still occurs with ZCULL accuracy set to approximate, unfortunately. Other settings for this test: RSX FIFO accuracy was set to Atomic and Driver Wake-Up Delay at 280. Tested against the latest commit 6bb01066fba5cf2b569d31b4ddad7caf235bfaa7. I don't believe there's any way to prevent this title from freezing, I've tried many different settings.

Here's a log from it freezing after about 30 minutes of play in zombie mode, single player: RPCS3.log.zip

kd-11 commented 6 days ago

Thanks for confirming. In that case, this likely has nothing to do with RSX anymore.

Shoegzer commented 5 days ago

Sure thing. Looking at the logs you'll see the same CELL_NET_CTL_ERROR_NOT_CONNECTED followed by Thread (hang detect) [0x0033d844]} PPU: 'sys_timer_usleep' aborted exceptions being thrown as noted above from older builds. I'm not sure if that's a clue as to the root cause, it may simply be a benign message about not connecting to network services, but it looks possible to me at least.