Dynarec is broken on PC

[bslenul]

Description

Tried a few games (Alien Resurrection, Alundra, Gran Turismo, Castlevania SotN, Crash Bandicoot, etc.) and they all crash at some point with dynarec ON. Either before reaching the title screen or when starting a new game.

Steps to reproduce

Start one of the games mentioned in the description above on PC (tested on Windows 10 and a Linux Mint VM) with dynarec ON, probably other games are affected, only tried these ones so far. Quickest test is Gran Turismo, it crashes on the "NOTICE" screen, like 10sec after booting the game.

When did the behavior start?

edit: probably incorrect bisect since it still crashes with previous commits on my Linux VM... I bisected the crashes, looks like it started with b19ddc7982ef5935a3ed54cbc8525a06b3639e00

Your device/OS/platform/architecture

Windows 10 and a Linux Mint VM, seems to work fine on my old Android 32bit phone.

Logs (enable file logging and set log levels to DEBUG for core and frontend)

retroarch.log

It also shows this in the log console which doesn't appear in the log file:

Threaded recompiler started with 1 workers.
Memory map is sub-par. Emitted code will be slow.
Lightrec RAM usage: IR 0 KiB, CODE 0 KiB, MIPS 0 KiB, TOTAL 5269 KiB, avg. IPI 0.000000
Threaded recompiler started with 1 workers.
Memory map is sub-par. Emitted code will be slow.

edit: gdb output: pcsx_gdb.txt

[bslenul]

Another issue related to dynarec on PC: with the latest working version (e8633a2e14027e4552940ef3e1c27c40b94c4870) the opening FMV in Gran Turismo doesn't play properly, it's super laggy and some artifacts are showing until it freezes and then it's skipped entirely to send you to the main menu.

I am really not sure of my bisect for this one, it appears to have started with 98fa08a56df29aeed65dfbc7697603c90bc66144 , it works fine with the previous commit (519a9ea123f92f3b02c059a32e4e9fd1170c8cbb) BUT since it couldn't compile due to missing lightning.h I had to apply d51bb08a1aedb5b20f0a73d101942825ceac5ef3 on top of it, so my bisect could be incorrect, not sure...

In any case, here's how it looks with e8633a2e14027e4552940ef3e1c27c40b94c4870:

https://user-images.githubusercontent.com/33353403/158055389-2f30d2bb-92c0-4f0f-af30-67e46a6a9c49.mp4

And how it looks with 519a9ea123f92f3b02c059a32e4e9fd1170c8cbb (+ d51bb08a1aedb5b20f0a73d101942825ceac5ef3):

https://user-images.githubusercontent.com/33353403/158055436-7ac6db07-9f2b-4577-951c-41e5c4ecfd6f.mp4

[bslenul]

Oh boy, I'm getting different results on my Linux VM, e8633a2e14027e4552940ef3e1c27c40b94c4870 crashes there -_-

519a9ea123f92f3b02c059a32e4e9fd1170c8cbb + d51bb08a1aedb5b20f0a73d101942825ceac5ef3 works fine on Linux too however, no crash and FMV playing properly.

So yeah, can't tell for sure where it started exactly, sorry :/

edit: gdb output using latest commit: pcsx_gdb_linux.txt

[Sanaki]

Gave bisecting this a spin myself. Results were... messy. A quick rundown excluding the intermediate commits to forks (testing EU Ape Escape in bin+cue+sbi format):

Last good: 7e3ce9afd1ddd5c16f20957cc7a46f6e681bff50 47e92264b0bcdd851e8b0973cd5036cc6ca8e4ba couldn't build due to missing lightning.h, fixed in next commit: e68dc4d6acd30112bf175243291e4198ce8003ca builds, has visual flaws (monkey helmets, visible on title screen), doesn't seem to crash when loaded through RetroArch GUI, instant segfault when CLI launched (ERROR: Cannot create recompiler: Out of memory) e8633a2e14027e4552940ef3e1c27c40b94c4870 identical to prior bb50ea2553de95dffecc24b9f41b423a8fc63142 crashing begins even when GUI loaded

Bisect done on x86_64 Linux. All commits after the one marked last good will segfault immediately when CLI loaded. I didn't test further commits past the final listed, since the CLI crash and the delayed GUI crash both still occur on current (75d5614bce62f7c2b3ca196574cf28bc9d050cdb). The visual issue mentioned is also still present on this commit.

[notaz]

Probably should just disable the dynarec on non-ARM as there are too many issues reported. @pcercuei

[pcercuei]

Ok, I wasn't aware of these issues (should have @'d me at the beginning).

Give me a couple of days and I'll fix them.

[pcercuei]

First broken commit is https://github.com/pcercuei/lightrec/commit/a8e021b

[pcercuei]

Please try https://github.com/libretro/pcsx_rearmed/pull/643. It does fix the opening of Gran Turismo here.

[bslenul]

No crash/issue with the few titles I mentioned in the issue, I could test a few more games later if needed, but looks good 👍

[bslenul]

Just booted my Linux Mint VM to do more tests, it still crashes there. Unfortunately I don't have a real Linux machine to test, so idk if it's just an issue with my VM or not.

[pcercuei]

"it still crashes there" isn't very helpful :)

[bslenul]

Yeah, it's not even a crash btw, like when the GT intro is about to start the screen stays black and RA just hangs (I can't open the Quick Menu or anything). I tried gdb but idk how to get anything useful since RA doesn't actually crash.

[hizzlekizzle]

Hmm, I'm not having any issues with Gran Turismo here. It's getting through the intro and into the game just fine for me using latest master. That's what I should be testing, right?

[Sanaki]

Just tested 37d9bf8315be570a350cd44876ae14f9b0eff20b on Linux x86_64. GUI-based crashing seems to be resolved. Gran Turismo seems to be fine (Rev 1 tested). The Ape Escape helmet effects are still broken with dynarec enabled, though that's a fairly unimportant issue compared to the crashes. The core also still immediately crashes when CLI launched with dynarec enabled.

CLI launch crash log and backtrace (debug build)

``` (gdb) run Starting program: /home/ccrowley/src/RetroArch/retroarch -L /home/ccrowley/src/libretro-super/libretro-pcsx_rearmed/pcsx_rearmed_libretro.so /home/ccrowley/ROMs/Sony\ -\ PlayStation/Other/Ape\ Escape\ \(Europe\)\ \(+sub\).chd [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [INFO] RetroArch 1.10.2 (Git 5d2b5e8722) [INFO] === Build ======================================= [INFO] CPU Model Name: Intel(R) Core(TM) i5-7300HQ CPU @ 2.50GHz [INFO] Capabilities: MMX MMXEXT SSE SSE2 SSE3 SSSE3 SSE4 SSE4.2 AES AVX AVX2 [INFO] Built: Apr 8 2022 [INFO] Version: 1.10.2 [INFO] Git: 5d2b5e8722 [INFO] ================================================= [New Thread 0x7fffe6b5e700 (LWP 25056)] [INFO] [Input]: Found input driver: "x". [INFO] [Core]: Loading dynamic libretro core from: "/home/ccrowley/src/libretro-super/libretro-pcsx_rearmed/pcsx_rearmed_libretro.so" [INFO] [Overrides]: No core-specific overrides found at "/home/ccrowley/.config/retroarch/config/PCSX-ReARMed/PCSX-ReARMed.cfg". [INFO] [Overrides]: No content-dir-specific overrides found at "/home/ccrowley/.config/retroarch/config/PCSX-ReARMed/Other.cfg". [INFO] [Overrides]: No game-specific overrides found at "/home/ccrowley/.config/retroarch/config/PCSX-ReARMed/Ape Escape (Europe) (+sub).cfg". [INFO] [Environ]: GET_LOG_INTERFACE. [INFO] [Environ]: SET_CONTROLLER_INFO. [INFO] Controller port: 1 [INFO] standard (ID: 1) [INFO] analog (ID: 261) [INFO] dualshock (ID: 517) [INFO] negcon (ID: 773) [INFO] guncon (ID: 260) [INFO] mouse (ID: 258) [INFO] (null) (ID: 0) [INFO] Controller port: 2 [INFO] standard (ID: 1) [INFO] analog (ID: 261) [INFO] dualshock (ID: 517) [INFO] negcon (ID: 773) [INFO] guncon (ID: 260) [INFO] mouse (ID: 258) [INFO] (null) (ID: 0) [INFO] Controller port: 3 [INFO] standard (ID: 1) [INFO] analog (ID: 261) [INFO] dualshock (ID: 517) [INFO] negcon (ID: 773) [INFO] guncon (ID: 260) [INFO] mouse (ID: 258) [INFO] (null) (ID: 0) [INFO] Controller port: 4 [INFO] standard (ID: 1) [INFO] analog (ID: 261) [INFO] dualshock (ID: 517) [INFO] negcon (ID: 773) [INFO] guncon (ID: 260) [INFO] mouse (ID: 258) [INFO] (null) (ID: 0) [INFO] Controller port: 5 [INFO] standard (ID: 1) [INFO] analog (ID: 261) [INFO] dualshock (ID: 517) [INFO] negcon (ID: 773) [INFO] guncon (ID: 260) [INFO] mouse (ID: 258) [INFO] (null) (ID: 0) [INFO] Controller port: 6 [INFO] standard (ID: 1) [INFO] analog (ID: 261) [INFO] dualshock (ID: 517) [INFO] negcon (ID: 773) [INFO] guncon (ID: 260) [INFO] mouse (ID: 258) [INFO] (null) (ID: 0) [INFO] Controller port: 7 [INFO] standard (ID: 1) [INFO] analog (ID: 261) [INFO] dualshock (ID: 517) [INFO] negcon (ID: 773) [INFO] guncon (ID: 260) [INFO] mouse (ID: 258) [INFO] (null) (ID: 0) [INFO] Controller port: 8 [INFO] standard (ID: 1) [INFO] analog (ID: 261) [INFO] dualshock (ID: 517) [INFO] negcon (ID: 773) [INFO] guncon (ID: 260) [INFO] mouse (ID: 258) [INFO] (null) (ID: 0) [INFO] [Environ]: GET_CORE_OPTIONS_VERSION. [INFO] [Environ]: GET_LANGUAGE: "0". [INFO] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_INTL. [INFO] [Remaps]: Remap directory: "/home/ccrowley/.config/retroarch/config/remaps". [INFO] [Remaps]: Core-specific remap found at "/home/ccrowley/.config/retroarch/config/remaps/PCSX-ReARMed/PCSX-ReARMed.rmp". [INFO] [Overrides]: Redirecting save file to "/home/ccrowley/.config/retroarch/saves/PCSX-ReARMed/Ape Escape (Europe) (+sub).srm". [INFO] [Overrides]: Redirecting save state to "/home/ccrowley/.config/retroarch/states/PCSX-ReARMed/Ape Escape (Europe) (+sub).state". [INFO] [Environ]: GET_MESSAGE_INTERFACE_VERSION. [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_memcard2 = "disabled" [libretro INFO] Memcard 2: disabled [libretro INFO] Starting PCSX-ReARMed 37d9bf8 [libretro INFO] Running PCSX Version 1.9 (Apr 8 2022). [New Thread 0x7fffe5ba0700 (LWP 25057)] [New Thread 0x7fffe539f700 (LWP 25058)] [New Thread 0x7fffe4b9e700 (LWP 25059)] Threaded recompiler started with 3 workers. Memory map is sub-par. Emitted code will be slow. [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_bios = "auto" [INFO] [Environ]: SYSTEM_DIRECTORY: "/home/ccrowley/.config/retroarch/system". [libretro INFO] found BIOS file: psxonpsp660.bin [INFO] [Environ]: GET_CAN_DUPE: true [INFO] [Environ]: GET_DISK_CONTROL_INTERFACE_VERSION. [INFO] [Environ]: SET_DISK_CONTROL_EXT_INTERFACE. [INFO] [Environ]: GET_RUMBLE_INTERFACE. [INFO] [Environ]: PERFORMANCE_LEVEL: 6. [INFO] [Content]: Content loading skipped. Implementation will load it on its own. [INFO] [Environ]: SET_INPUT_DESCRIPTORS: [INFO] RetroPad, Port 1, Button "B (bottom)" => "Cross" [INFO] RetroPad, Port 1, Button "Y (left)" => "Square" [INFO] RetroPad, Port 1, Button "Select" => "Select" [INFO] RetroPad, Port 1, Button "Start" => "Start" [INFO] RetroPad, Port 1, Button "D-Pad Up" => "D-Pad Up" [INFO] RetroPad, Port 1, Button "D-Pad Down" => "D-Pad Down" [INFO] RetroPad, Port 1, Button "D-Pad Left" => "D-Pad Left" [INFO] RetroPad, Port 1, Button "D-Pad Right" => "D-Pad Right" [INFO] RetroPad, Port 1, Button "A (right)" => "Circle" [INFO] RetroPad, Port 1, Button "X (up)" => "Triangle" [INFO] RetroPad, Port 1, Button "L" => "L1" [INFO] RetroPad, Port 1, Button "R" => "R1" [INFO] RetroPad, Port 1, Button "L2" => "L2" [INFO] RetroPad, Port 1, Button "R2" => "R2" [INFO] RetroPad, Port 1, Button "L3" => "L3" [INFO] RetroPad, Port 1, Button "R3" => "R3" [INFO] RetroPad, Port 2, Button "B (bottom)" => "Cross" [INFO] RetroPad, Port 2, Button "Y (left)" => "Square" [INFO] RetroPad, Port 2, Button "Select" => "Select" [INFO] RetroPad, Port 2, Button "Start" => "Start" [INFO] RetroPad, Port 2, Button "D-Pad Up" => "D-Pad Up" [INFO] RetroPad, Port 2, Button "D-Pad Down" => "D-Pad Down" [INFO] RetroPad, Port 2, Button "D-Pad Left" => "D-Pad Left" [INFO] RetroPad, Port 2, Button "D-Pad Right" => "D-Pad Right" [INFO] RetroPad, Port 2, Button "A (right)" => "Circle" [INFO] RetroPad, Port 2, Button "X (up)" => "Triangle" [INFO] RetroPad, Port 2, Button "L" => "L1" [INFO] RetroPad, Port 2, Button "R" => "R1" [INFO] RetroPad, Port 2, Button "L2" => "L2" [INFO] RetroPad, Port 2, Button "R2" => "R2" [INFO] RetroPad, Port 2, Button "L3" => "L3" [INFO] RetroPad, Port 2, Button "R3" => "R3" [INFO] RetroPad, Port 3, Button "B (bottom)" => "Cross" [INFO] RetroPad, Port 3, Button "Y (left)" => "Square" [INFO] RetroPad, Port 3, Button "Select" => "Select" [INFO] RetroPad, Port 3, Button "Start" => "Start" [INFO] RetroPad, Port 3, Button "D-Pad Up" => "D-Pad Up" [INFO] RetroPad, Port 3, Button "D-Pad Down" => "D-Pad Down" [INFO] RetroPad, Port 3, Button "D-Pad Left" => "D-Pad Left" [INFO] RetroPad, Port 3, Button "D-Pad Right" => "D-Pad Right" [INFO] RetroPad, Port 3, Button "A (right)" => "Circle" [INFO] RetroPad, Port 3, Button "X (up)" => "Triangle" [INFO] RetroPad, Port 3, Button "L" => "L1" [INFO] RetroPad, Port 3, Button "R" => "R1" [INFO] RetroPad, Port 3, Button "L2" => "L2" [INFO] RetroPad, Port 3, Button "R2" => "R2" [INFO] RetroPad, Port 3, Button "L3" => "L3" [INFO] RetroPad, Port 3, Button "R3" => "R3" [INFO] RetroPad, Port 4, Button "B (bottom)" => "Cross" [INFO] RetroPad, Port 4, Button "Y (left)" => "Square" [INFO] RetroPad, Port 4, Button "Select" => "Select" [INFO] RetroPad, Port 4, Button "Start" => "Start" [INFO] RetroPad, Port 4, Button "D-Pad Up" => "D-Pad Up" [INFO] RetroPad, Port 4, Button "D-Pad Down" => "D-Pad Down" [INFO] RetroPad, Port 4, Button "D-Pad Left" => "D-Pad Left" [INFO] RetroPad, Port 4, Button "D-Pad Right" => "D-Pad Right" [INFO] RetroPad, Port 4, Button "A (right)" => "Circle" [INFO] RetroPad, Port 4, Button "X (up)" => "Triangle" [INFO] RetroPad, Port 4, Button "L" => "L1" [INFO] RetroPad, Port 4, Button "R" => "R1" [INFO] RetroPad, Port 4, Button "L2" => "L2" [INFO] RetroPad, Port 4, Button "R2" => "R2" [INFO] RetroPad, Port 4, Button "L3" => "L3" [INFO] RetroPad, Port 4, Button "R3" => "R3" [INFO] RetroPad, Port 5, Button "B (bottom)" => "Cross" [INFO] RetroPad, Port 5, Button "Y (left)" => "Square" [INFO] RetroPad, Port 5, Button "Select" => "Select" [INFO] RetroPad, Port 5, Button "Start" => "Start" [INFO] RetroPad, Port 5, Button "D-Pad Up" => "D-Pad Up" [INFO] RetroPad, Port 5, Button "D-Pad Down" => "D-Pad Down" [INFO] RetroPad, Port 5, Button "D-Pad Left" => "D-Pad Left" [INFO] RetroPad, Port 5, Button "D-Pad Right" => "D-Pad Right" [INFO] RetroPad, Port 5, Button "A (right)" => "Circle" [INFO] RetroPad, Port 5, Button "X (up)" => "Triangle" [INFO] RetroPad, Port 5, Button "L" => "L1" [INFO] RetroPad, Port 5, Button "R" => "R1" [INFO] RetroPad, Port 5, Button "L2" => "L2" [INFO] RetroPad, Port 5, Button "R2" => "R2" [INFO] RetroPad, Port 5, Button "L3" => "L3" [INFO] RetroPad, Port 5, Button "R3" => "R3" [INFO] [Environ]: SET_PIXEL_FORMAT: RGB565. [libretro INFO] RGB565 supported, using it [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_frameskip_type = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_frameskip_threshold = "33" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_frameskip_interval = "3" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_region = "auto" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_multitap = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_negcon_deadzone = "0" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_negcon_response = "linear" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_analog_axis_modifier = "circle" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_vibration = "enabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_dithering = "enabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_duping_enable = "enabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_display_internal_fps = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_drc = "enabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_spu_reverb = "enabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_spu_interpolation = "simple" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_pe2_fix = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_icache_emulation = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_inuyasha_fix = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_async_cd = "sync" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_noxadecoding = "enabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_nocdaudio = "enabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_spuirq = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_thread_rendering = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_peops_odd_even_bit = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_peops_expand_screen_width = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_peops_ignore_brightness = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_peops_disable_coord_check = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_peops_lazy_screen_update = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_peops_old_frame_skip = "enabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_peops_repeated_triangles = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_peops_quads_with_triangles = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gpu_peops_fake_busy_state = "disabled" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_show_gpu_peops_settings = "enabled" [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gunconadjustx = "0" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gunconadjusty = "0" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gunconadjustratiox = "1" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_gunconadjustratioy = "1" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_nostalls = "N/A" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_input_sensitivity = "1.00" [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_show_other_input_settings = "enabled" [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [DEBUG] [Environ]: RETRO_ENVIRONMENT_SET_CORE_OPTIONS_DISPLAY. [INFO] [Environ]: GET_VARIABLE: pcsx_rearmed_show_bios_bootlogo = "disabled" [libretro INFO] plugin: plugins/builtin_gpu [libretro INFO] plugin: plugins/builtin_spu [libretro INFO] plugin: plugins/builtin_pad [libretro INFO] plugin: plugins/builtin_pad [New Thread 0x7fffe439d700 (LWP 25060)] [libretro INFO] Plugins loaded. [libretro INFO] chd: TRACK:1 TYPE:MODE2_RAW SUBTYPE:RW_RAW FRAMES:181632 PREGAP:0 PGTYPE:MODE1 PGSUB:NONE POSTGAP:0 [libretro INFO] Loaded CD Image: /home/ccrowley/ROMs/Sony - PlayStation/Other/Ape Escape (Europe) (+sub).chd[+chd]. [libretro INFO] Track 01 (DATA) - Start 00:02:00, Length 40:21:57 [libretro INFO] CD-ROM Label: SCES01564 [libretro INFO] CD-ROM ID: SCES01564 [libretro INFO] CD-ROM EXE Name: SCES_015.64;1 Lightrec RAM usage: IR 0 KiB, CODE 0 KiB, MIPS 0 KiB, TOTAL 5309 KiB, avg. IPI 0.000000 [Thread 0x7fffe4b9e700 (LWP 25059) exited] [Thread 0x7fffe539f700 (LWP 25058) exited] [Thread 0x7fffe5ba0700 (LWP 25057) exited] ERROR: Cannot create recompiler: Out of memory Thread 1 "retroarch" received signal SIGSEGV, Segmentation fault. 0x00007fffe5c34102 in lightrec_plugin_reset () at libpcsxcore/lightrec/plugin.c:551 551 regs->cp0[12] = 0x10900000; // COP0 enabled | BEV = 1 | TS = 1 (gdb) thread apply all bt Thread 6 (Thread 0x7fffe439d700 (LWP 25060) "retroarch"): #0 0x00007ffff46da7c6 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x7fffe60d9928 ) at ../sysdeps/unix/sysv/linux/futex-internal.h:205 #1 do_futex_wait (sem=sem@entry=0x7fffe60d9928 , abstime=0x0) at sem_waitcommon.c:111 #2 0x00007ffff46da8b8 in __new_sem_wait_slow (sem=0x7fffe60d9928 , abstime=0x0) at sem_waitcommon.c:181 #3 0x00007fffe5c8fb6d in spu_worker_thread (unused=0x0) at plugins/dfsound/spu.c:1390 #4 0x00007ffff46d16db in start_thread (arg=0x7fffe439d700) at pthread_create.c:463 #5 0x00007fffef29d61f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 2 (Thread 0x7fffe6b5e700 (LWP 25056) "retroarch"): #0 0x00007ffff46d7ad3 in futex_wait_cancelable (private=, expected=0, futex_word=0x555556a2b628) at ../sysdeps/unix/sysv/linux/futex-internal.h:88 #1 __pthread_cond_wait_common (abstime=0x0, mutex=0x555556a2ada0, cond=0x555556a2b600) at pthread_cond_wait.c:502 #2 __pthread_cond_wait (cond=0x555556a2b600, mutex=0x555556a2ada0) at pthread_cond_wait.c:655 #3 0x0000555555a4a4c1 in scond_wait (cond=0x555556a2b600, lock=0x555556a2ada0) at libretro-common/rthreads/rthreads.c:735 #4 0x0000555555685021 in threaded_worker (userdata=0x0) at libretro-common/queues/task_queue.c:489 #5 0x0000555555a4a092 in thread_wrap (data_=0x555556a2b6f0) at libretro-common/rthreads/rthreads.c:143 #6 0x00007ffff46d16db in start_thread (arg=0x7fffe6b5e700) at pthread_create.c:463 #7 0x00007fffef29d61f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 1 (Thread 0x7ffff7dfe800 (LWP 25050) "retroarch"): #0 0x00007fffe5c34102 in lightrec_plugin_reset () at libpcsxcore/lightrec/plugin.c:551 #1 0x00007fffe5c04ec1 in psxReset () at libpcsxcore/r3000a.c:64 #2 0x00007fffe5bfc1c3 in EmuReset () at libpcsxcore/psxcommon.c:41 #3 0x00007fffe5cfd029 in SysReset () at frontend/main.c:739 #4 0x00007fffe5cf8812 in retro_load_game (info=0x555556a4c9e0) at frontend/libretro.c:1478 #5 0x0000555555668ceb in core_load_game (load_info=0x7fffffffa2b0) at runloop.c:8156 #6 0x0000555555687c7b in content_file_load (p_content=0x5555568dc318 , content=0x7fffffffa7b0, content_ctx=0x7fffffffa7d0, error_enum=0x7fffffffa76c, error_string=0x7fffffffa770, special=0x0) at tasks/task_content.c:1156 #7 0x00005555556882b9 in content_file_init (content_ctx=0x7fffffffa7d0, p_content=0x5555568dc318 , content=0x7fffffffa7b0, error_enum=0x7fffffffa76c, error_string=0x7fffffffa770) at tasks/task_content.c:1339 #8 0x000055555568b7cc in content_init () at tasks/task_content.c:3091 #9 0x0000555555661b6e in event_init_content (settings=0x7ffff7fb0010, input_st=0x555556930920 ) at runloop.c:5100 #10 0x0000555555662480 in runloop_event_init_core (settings=0x7ffff7fb0010, input_data=0x555556930920 , type=CORE_TYPE_PLAIN) at runloop.c:5423 #11 0x000055555564e731 in command_event (cmd=CMD_EVENT_CORE_INIT, data=0x5555568e3954 ) at retroarch.c:2389 #12 0x0000555555655d85 in retroarch_main_init (argc=4, argv=0x7fffffffd6e8) at retroarch.c:5392 #13 0x0000555555688a65 in content_load (info=0x7fffffffd5b0, p_content=0x5555568dc318 ) at tasks/task_content.c:1482 #14 0x000055555568aa1c in task_load_content_internal (content_info=0x7fffffffd5b0, loading_from_menu=true, loading_from_cli=true, loading_from_companion_ui=false) at tasks/task_content.c:2604 #15 0x000055555568ad03 in task_push_load_content_from_cli (core_path=0x0, fullpath=0x0, content_info=0x7fffffffd5b0, type=CORE_TYPE_PLAIN, cb=0x0, user_data=0x0) at tasks/task_content.c:2696 #16 0x0000555555651138 in rarch_main (argc=4, argv=0x7fffffffd6e8, data=0x0) at retroarch.c:3851 #17 0x0000555555894b95 in main (argc=4, argv=0x7fffffffd6e8) at ui/drivers/ui_qt.cpp:4326 ```

[bslenul]

OK good to know, probably just an issue with my VM then!

That's what I should be testing, right?

Yeah, for me RA hangs just before the GT intro is supposed to play. And RA hangs directly on launch with Alien Resurrection. Alundra and Crash Bandicoot seem to work fine now however.

But on Windows no issue, I tried a few more games yesterday and they all went in-game. Even tried to reproduce the crash when booting via CLI, but it starts fine on Windows.

[pcercuei]

Alien Resurrection boots and works fine here.

[pcercuei]

@Sanaki for your CLI build make sure you rebuild everything from scratch, it looks like you're using an older Lightrec with the updated pcsx_libretro core or vice-versa.

[bslenul]

I get random crashes on startup on Windows (normal boot, not CLI): crash-220409-130153.log

That was after booting ~10 games in a row but sometimes it happens on the very first game, seems completely random, I could even boot ~20 games in a row without a crash at some point... When it crashes I get "ERROR: Cannot create recompiler: Out of memory" in the log console window.

As for CLI on Linux, I can reproduce the crash, here's the output with RA built with DEBUG=1 SANITIZER=thread:

ERROR: Cannot create recompiler: Out of memory
ThreadSanitizer:DEADLYSIGNAL
==2441==ERROR: ThreadSanitizer: SEGV on unknown address 0x00000000020c (pc 0x7f6925d20371 bp 0x7ffdf8f7fe50 sp 0x7ffdf8f7fe40 T2441)
==2441==The signal is caused by a READ memory access.
==2441==Hint: address points to the zero page.
    #0 lightrec_destroy deps/lightrec/lightrec.c:1582 (pcsx_rearmed_libretro.so+0xe3371)
    #1 lightrec_plugin_shutdown libpcsxcore/lightrec/plugin.c:539 (pcsx_rearmed_libretro.so+0x970f3)
    #2 lightrec_plugin_reset libpcsxcore/lightrec/plugin.c:546 (pcsx_rearmed_libretro.so+0x97107)
    #3 psxReset libpcsxcore/r3000a.c:64 (pcsx_rearmed_libretro.so+0x67eae)
    #4 EmuReset libpcsxcore/psxcommon.c:41 (pcsx_rearmed_libretro.so+0x5efca)
    #5 SysReset frontend/main.c:739 (pcsx_rearmed_libretro.so+0x160d30)
    #6 retro_load_game frontend/libretro.c:1478 (pcsx_rearmed_libretro.so+0x15c601)
    #7 core_load_game /home/bobby/Documents/RetroArch/runloop.c:8188 (retroarch_asan_thread+0x1301e0)
    #8 content_file_load tasks/task_content.c:1156 (retroarch_asan_thread+0x156840)
    #9 content_file_init tasks/task_content.c:1339 (retroarch_asan_thread+0x1571be)
    #10 content_init tasks/task_content.c:2958 (retroarch_asan_thread+0x15c798)
    #11 event_init_content /home/bobby/Documents/RetroArch/runloop.c:5111 (retroarch_asan_thread+0x12494f)
    #12 runloop_event_init_core /home/bobby/Documents/RetroArch/runloop.c:5434 (retroarch_asan_thread+0x125778)
    #13 command_event /home/bobby/Documents/RetroArch/retroarch.c:2386 (retroarch_asan_thread+0x1083a7)
    #14 retroarch_main_init /home/bobby/Documents/RetroArch/retroarch.c:5388 (retroarch_asan_thread+0x11185d)
    #15 content_load tasks/task_content.c:1482 (retroarch_asan_thread+0x157e4c)
    #16 task_load_content_internal tasks/task_content.c:2471 (retroarch_asan_thread+0x15b011)
    #17 task_push_load_content_from_cli tasks/task_content.c:2563 (retroarch_asan_thread+0x15b42c)
    #18 rarch_main /home/bobby/Documents/RetroArch/retroarch.c:3847 (retroarch_asan_thread+0x10c435)
    #19 main ui/drivers/ui_qt.cpp:4326 (retroarch_asan_thread+0x3d51e3)
    #20 __libc_start_main <null> (libc.so.6+0x240b2)
    #21 _start <null> (retroarch_asan_thread+0xf794d)

ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV deps/lightrec/lightrec.c:1582 in lightrec_destroy
==2441==ABORTING

[pcercuei]

The sigsegv you see is probably just a consequence of the first "Out of memory" error. The glue code calls lightrec_init() without bothering to check if there's an error.

Now, about the out-of-memory error... Could you check the return value of get_processors_count() in lightrec_recompiler_init() in recompiler.c? I think in your case it returns a negative value cast to unsigned, which causes Lightrec to try to allocate an immense amount of RAM, which fails.

[bslenul]

"nb_cpus" from "lightrec_recompiler_init()" is "4" on my Linux VM (either from normal boot or from CLI), "1" on Windows.

[pcercuei]

Try to force e.g. nb_cpus = 4 and see if it fixes the problem.

[bslenul]

No difference, still crashes with CLI on Linux, and I was able to get a random crashes on startup on Windows.

Even got 2 crashes in a row on Windows with 2 different crash logs oO 1st was the same as the one I posted above, here's the 2nd (which look very similar to the gdb output posted by Sanaki): crash-220409-162835.log

[Sanaki]

@Sanaki for your CLI build make sure you rebuild everything from scratch, it looks like you're using an older Lightrec with the updated pcsx_libretro core or vice-versa.

I rebuilt latest master with ccache disabled. Same result. Lightrec is the version in the master branch (37d9bf8315be570a350cd44876ae14f9b0eff20b). It's not a submodule. Build command was simply DEBUG=1 DYNAREC=lightrec make -f Makefile.libretro -j4. Same result without DEBUG=1, just a less useful backtrace. If you need specific details, I'm happy to provide them.

[pcercuei]

@Sanaki try forcing nb_cpus = 4 in lightrec's recompiler.c, see if it works better.

[Sanaki]

Still instantly segfaults on CLI launch with an out of memory error. Same result.

[pcercuei]

Nevermind. Found it.

[pcercuei]

Try with this fix: https://github.com/pcercuei/lightrec/commit/b9f1b040bb1333508caee3795375e13b8cf8f684

[Sanaki]

That fixed it.

[pcercuei]

Ok, I'll send a new PR then.

[pcercuei]

Anything else?

[Sanaki]

Hmm... just ran a few quick tests to make sure and the behavior was momentarily inconsistent. After a few rapid-fire launches it runs fine every time now, so... I have no idea what to make of that. Some edge race condition maybe? Might be safe to shrug off. Better to report just in case it crops up again though.

Command run: retroarch -L pcsx_rearmed "Ape Escape (Europe) (+sub).chd"

Result 1: Game launches, all is well. This is now the only result I can get, but...

Result 2:

Threaded recompiler started with 3 workers.
Memory map is sub-par. Emitted code will be slow.
ERROR: Segmentation fault in recompiled code: invalid load/store at address 0x2003e1f4
ERROR: Was executing block PC 0x000013c8
Exiting at cycle 0x00040728

Result 3:

Threaded recompiler started with 3 workers.
Memory map is sub-par. Emitted code will be slow.
Segmentation fault (core dumped)

Result 4: Following the memory map is sub-par line, this line repeating every 1.5 seconds or so until killed, no launch: Lightrec RAM usage: IR 22 KiB, CODE 80 KiB, MIPS 11 KiB, TOTAL 5423 KiB, avg. IPI 7.079631

EDIT: Haven't managed to trigger the issue again after disabling ccache and rebuilding, so hopefully it was just a stale cache issue. Issue -should- be safe to close.

[pcercuei]

@Sanaki ok, please open a new bug report if it happens again.