Eternal Darkness regression, freezes on save/load state

bslenul commented 4 years ago

edit: Black screen was fixed, but the game freezes on save/load state.

Hello! Eternal Darkness is pretty much unplayable with latest commit (f4f4947 when typing this), ingame is just a black screen (except when opening inventory) :(

Here's a video comparing 1fbd599 and f4f4947: https://youtu.be/M1KuFd5lUso

I'm loading the game, then I go to ingame "Options" menu, with the old core no issue, with new you can see that the transitions are very laggy. Also I obviously can't show that on the video, but just pressing up/down in the menu feels really unresponsive.
Then I load my save, with old core no issue, with new however the loading screen only appears when the loading is actually finished, and then ingame it's just a black screen (I can hear my character moving, etc. in the background), for some reason opening the inventory works fine. With Vulkan I can see this line appearing over and over again while having the black screen: [libretro WARN] 01:32:705 VideoBackends\Vulkan\VertexManager.cpp:152 W[Video]: Executing command list while waiting for space in vertex/index buffer.

Video taken on Windows, but it's pretty much the same behavior on my Linux VM. No issue with standalone up-to-date. Not sure if other games are affected by this, I only have ~20 GC/Wii games and so far it's the only one that gave me trouble compared to old core.

Mr-Jake commented 4 years ago

A search reveals a black screen is a common problem for this game.

The Dolphin forums suggest changing the settings to Single CPU Core and enable Deterministic GPU. It was found Dualcore and SyncGPU modes could cause the black screen.

Not sure if this will help, but it's worth a try.

bslenul commented 4 years ago

Thanks for the reply! So based on that link and what you said, I tried this:

[Core]
CPUThread = False
SyncGPU = False

The first line crashes RA when loading the game and the second didn't help unfortunately. GPUDeterminismMode = fake-completion is already set by default for the game.

i30817 commented 4 years ago

I have this on /home/i30817/.config/retroarch/system/dolphin-emu/Sys/GameSettings/GED.ini

# GEDE01, GEDP01 - Eternal Darkness

[Core]
# hang in edgar alan poe quote at start
GPUDeterminismMode = fake-completion

[OnLoad]
# Add memory patches to be loaded once on boot here.

[OnFrame]
# Add memory patches to be applied every frame here.

[ActionReplay]
# Add action replay cheats here.

[Video_Enhancements]
# low quality mipmap errors. May cause problems on graphics if any "mip trick" effects are used by the game
ArbitraryMipmapDetection = False

[Video_Hacks]
#Green / Purple Transitions if you have hybrid xfb turned on. If it's disabled turn on real xfb instead
XFBToTextureEnable = False

I edited it based on the wiki page for eternal darkness recommendations. I also need to use glcore not gl (i don't have vulkan capable gpu). Otherwise i get a black screen and heavily distorted core menu, with black borders around, though i suppose it might be different for you if you allow RA cores to not use the configured driver automatically (i don't like not knowing what driver is being tried).

edit: if you can't access the menu by pressing 'start' key see if you can map 'select' to 'start' and start it that way (if you're not using a gc controller). I'm having problems making the game recognize start, which opens the game menu in the updated core, but setting up 'select' starts it.

Maybe it's because i'm using a keyboard as retropad, maybe something else, but looks like a bug.

bslenul commented 4 years ago

Thanks, I already have XFBToTextureEnable = False for the purple screen transitions bug, I tried ArbitraryMipmapDetection = False but it didn't change anything.

Tried "glcore" instead of "vulkan", no changes but it spams this in logs:

[libretro WARN] 45:36:544 VideoBackends\OGL\Render.cpp:111 W[Host GPU]: id: 20052, source: API, type: Performance - Pixel-path performance warning: Pixel transfer is synchronized with 3D rendering.

i30817 commented 4 years ago

Did you make sure in the retroarch options that the cores can't change the driver option?

I have: settings->core->hardware shared context = OFF settings->core->allow cores to switch the video driver = OFF

Then if a core really really needs one driver i force it by loading the core/game, going into the menu, changing the driver, going back to the quick menu and saving the changes from default as a 'core option'.

bslenul commented 4 years ago

Yes, it was using glcore for sure.

Just to be clear this is a new bug with recent commits (since rebase maybe?), I have like 3 hours of playtime in the game and haven't had any issue (well except the purple transitions but that's fixed with the XBF to textures thing) with 1fbd599 on Windows (as you can see in the beginning of the video I made), also tried with 2dd20d2 on my Linux VM (the one currently available in the core updater), and no black screen either.

i30817 commented 4 years ago

I see. I'm building the current core now with

cmake -DENABLE_QT=OFF -DLIBRETRO=ON -DUSE_DISCORD_PRESENCE=OFF -DENABLE_TESTS=OFF -DENABLE_LTO=ON

And i'll see if the same thing happens. Maybe the upstream upgraded the opengl requirements and we don't have the cards for it (hopefully not).

i30817 commented 4 years ago

Yo. I compiled this and the core is too slow for me to use here right now. I did get in game but it was as if i was on a 1 frame per minute slideshow (and it often hanged), with a lot of 'empty' frames. I didn't see the same effect as your video, but there was surely something wrong, i doubt it can be otherwise.

bslenul commented 4 years ago

Yeah I guess "black screen" isn't really true, it depends, for example if I start a new game I'm stuck with the latest image of the intro and I can hear the gameplay behind, when I load a save I'm stuck with the loading screen and sometimes I get a new still image after a while :p And if I pause/unpause like on the video it's just a black screen.

i30817 commented 4 years ago

I'm currently building dolphin nogui upstream and putting it in action with this PR and this 'core' (after editing it to use the new drm platform and hardcoding the dolphin path because i didn't want to deal with installing a modification).

It has ... other problems (i added a patch to the pr comments but it still doesn't read keys combinations with modifiers like ctrl as modified keys just the original). It sapped my enthusiasm not being able to save without remapping 10 buttons and not 'solving it', how weird.

Well anyway, probably not what you were looking for but i find it telling that even after the big fat 1 minute warning of 'your driver is terrible and you should feel bad' on the core start, it's still much much faster than the current master here. Somethings are broken indeed.

bslenul commented 4 years ago

Hey!

Just tried the latest commit (W10):

With Vulkan: it crashes RA completely on launch: crash-201004-190117.log And the regular logs: retroarch2020_10_0419_07_51.log

With GLcore I can't get a crash log since it just "hangs" RA after a few seconds of black screen (I can hear the game during these few seconds), it spams stuff like crazy in regular logs, mostly these ones:

[libretro ERROR] 03:51:870 VideoBackends\OGL\ProgramShaderCache.cpp:399 E[Video]: Program failed linking:
[INFO] [GL debug (Low, API, Other)]: Texture state usage warning: The texture object (0) bound to texture image unit 1 does not have a defined base level and cannot be used for texture mapping.
[libretro WARN] 03:55:649 VideoBackends\OGL\Render.cpp:111 W[Host GPU]: id: 20052, source: API, type: Performance - Pixel-path performance warning: Pixel transfer is synchronized with 3D rendering.

retroarch2020_10_0419_03_46.log

I tried on my Linux VM but all games crash with ASAN, not only Eternal Darkness, so I'm not sure if this will be helpful or not:

==32139==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new [] vs operator delete) on 0x62e000000400
    #0 0x7f2b8c4c88df in operator delete(void*) (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x1108df)
    #1 0x7f2b5e442e6b in DPL2FSDecoder::~DPL2FSDecoder() (/home/bobby/.config/retroarch/cores/dolphin_libretro.so+0x458e6b)
    #2 0x7f2b5e43ad35 in AudioCommon::SurroundDecoder::~SurroundDecoder() (/home/bobby/.config/retroarch/cores/dolphin_libretro.so+0x450d35)
    #3 0x7f2b5e43aa1b in Mixer::~Mixer() (/home/bobby/.config/retroarch/cores/dolphin_libretro.so+0x450a1b)
    #4 0x7f2b5e43b478 in NullSound::~NullSound() (/home/bobby/.config/retroarch/cores/dolphin_libretro.so+0x451478)
    #5 0x7f2b5e108c76 in retro_run (/home/bobby/.config/retroarch/cores/dolphin_libretro.so+0x11ec76)
    #6 0x556e262ecaee in core_run /home/bobby/Documents/RetroArch/retroarch.c:40738
    #7 0x556e262e66c7 in runloop_iterate /home/bobby/Documents/RetroArch/retroarch.c:40103
    #8 0x556e261ebd32 in rarch_main /home/bobby/Documents/RetroArch/retroarch.c:17829
    #9 0x556e267148ff in main ui/drivers/qt/ui_qt_application.cpp:151
    #10 0x7f2b882440b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #11 0x556e2614f61d in _start (/home/bobby/Documents/RetroArch/retroarch+0x4ba061d)

0x62e000000400 is located 0 bytes inside of 41248-byte region [0x62e000000400,0x62e00000a520)
allocated by thread T0 (Video thread) here:
    #0 0x7f2b8c4c7b47 in operator new[](unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10fb47)
    #1 0x7f2b5e448e6c in kiss_fftr_alloc (/home/bobby/.config/retroarch/cores/dolphin_libretro.so+0x45ee6c)

SUMMARY: AddressSanitizer: alloc-dealloc-mismatch (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x1108df) in operator delete(void*)
==32139==HINT: if you don't care about these errors you may set ASAN_OPTIONS=alloc_dealloc_mismatch=0
==32139==ABORTING

I tried gdb too (with RA compiled with DEBUG=1 GL_DEBUG=1) but it only gives me this:

Thread 12 "CPU thread" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffacfb2700 (LWP 3077)]
0x00007fffacfb86c0 in ?? ()
(gdb) bt full
#0  0x00007fffacfb86c0 in ?? ()
No symbol table info available.
#1  0x0000000000000000 in ?? ()
No symbol table info available.

Which doesn't seem really helpful... Without ASAN or gdb, other games work fine but Eternal Darkness crashes. Logs while trying to run the game: retroarch2020_10_0501_08_08.log

I'm not a dev and I only know some basic stuff for debugging so I hope there's at least something helpful in my post :p

aliaspider commented 4 years ago

you need to disable fastmem when running through gdb, there should be a core option for it.

did you clean up your game settings ? might be best to delete the old Sys directory from system/dolphin-emu/ and copy over a new one.

bslenul commented 4 years ago

OK, gdb with fastmem disabled (every other core options are defaults):

Thread 12 "CPU thread" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffacfb2700 (LWP 3156)]
__memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:240
240 ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
(gdb) bt full
#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:240
No locals.
#1  0x00007fffca3ccd4d in OGL::ProgramShaderCache::UploadConstants(void const*, unsigned int) () from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#2  0x00007fffca403731 in VideoCommon::PostProcessing::BlitFromTexture(MathUtil::Rectangle<int> const&, MathUtil::Rectangle<int> const&, AbstractTexture const*, int) ()
   from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#3  0x00007fffca40b041 in Renderer::RenderXFBToScreen(MathUtil::Rectangle<int> const&, AbstractTexture const*, MathUtil::Rectangle<int> const&) () from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#4  0x00007fffca40efb9 in Renderer::Swap(unsigned int, unsigned int, unsigned int, unsigned int, unsigned long) () from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#5  0x00007fffca3e5ca4 in AsyncRequests::HandleEvent(AsyncRequests::Event const&) () from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#6  0x00007fffca3e615b in AsyncRequests::PushEvent(AsyncRequests::Event const&, bool) () from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#7  0x00007fffca44d104 in VideoBackendBase::Video_BeginField(unsigned int, unsigned int, unsigned int, unsigned int, unsigned long) () from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#8  0x00007fffca240d8a in VideoInterface::Update(unsigned long) () from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#9  0x00007fffca23dced in SystemTimers::(anonymous namespace)::VICallback(unsigned long, long) () from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#10 0x00007fffca1bb5c5 in CoreTiming::Advance() () from /home/bobby/.config/retroarch/cores/dolphin_libretro.so
No symbol table info available.
#11 0x00007fffaffbb02e in ?? ()
No symbol table info available.
#12 0x0000000000000000 in ?? ()
No symbol table info available.

That's with a fresh "Sys" folder (deleted the old one completely, not just overwritten).

aliaspider commented 4 years ago

just pushed a new commit, can you try it out ?

bslenul commented 4 years ago

Looking good!

No crash, no black screen, no menu at 5fps, etc. with glcore, vulkan and d3d11 on Windows 👍 I'm only able to test glcore on my Linux VM because vulkan isn't supported, but it works fine too!

The only thing I noticed is the spam in the log file with glcore (only on Windows): [libretro WARN] 03:29:399 VideoBackends\OGL\Render.cpp:111 W[Host GPU]: id: 20052, source: API, type: Performance - Pixel-path performance warning: Pixel transfer is synchronized with 3D rendering., I have almost 2k lines of that in a minute or less :x

But anyway, thank you! :D

aliaspider commented 4 years ago

hmm another windows only thing, not getting any log spam here. I'll have to check that out first before closing this.

bslenul commented 4 years ago

I had some time to mess a bit more with the core, save/load state insta-hang RetroArch in Eternal Darkness (on both Windows and Linux) :( Unfortunately since it just "hangs" I don't get anything with retroarch_debug.exe on Windows or gdb on Linux :/

But besides the states, I played a couple of chapters and it ran perfectly fine.

bslenul commented 4 years ago

Oh and about the spam in logs, for some weird reason it seems to only affect the first launch of the game!

So basically:

I start RetroArch and launch the game: spam.
Close content and launch the game again: no spam anymore!
Close RetroArch, restart it and restart the game: spam again.

And I noticed the same issue with Mario Sunshine but with Vulkan, it spams [libretro ERROR] 21:55:988 VideoBackends\Vulkan\VulkanLoader.cpp:220 E[Video]: (ReadbackQueries) vkGetQueryPoolResults failed: (1: VK_NOT_READY) but ONLY on its first launch, if I close content and restart the game no more spam. And it will start spamming again on first launch next time I'll restart RetroArch.

Dgdiniz commented 1 week ago

Hi. Here I can reproduce this save state freeze. Every time I try to save with F2, retroarch hangs. I tried a lot of configurations, but nothing. Any ideas?

libretro / dolphin

Eternal Darkness regression, freezes on save/load state #167