Libretro build segfaulting on multiple platforms

[Sanaki]

Thus far confirmed on 64-bit Windows and 64-bit Linux, presumably it affects other versions as well.

Bisect results as follows (reordered chronologically), there were a number of commits I couldn't build surrounding the issue:

There are only 'skip'ped commits left to test. The first bad commit could be any of: 4c46867230c548730550d6d52378cc7c267d577c 2da977bed25297a01c6589fd640fd65def3eb60b 34f6d085a5b7ece43f75415f3d2930df7793ac6e 1bcafafdbb52e62ab93f81eb7c0488456564c94a 481181714b3412af2994509b70555cb968385f0c 667a8522ccad57f7ef7ae3d860f3f73663044c2e 1af3009e80e115e496cfa6da50eada1f5abe3ca1 4b4e3432cde2493d6d9bdd8cd3da559ae57e6a16

We cannot bisect more!

Tail of strace:

stat("/home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so", {st_mode=S_IFREG|0755, st_size=28513544, ...}) = 0
stat("/home/ccrowley/.config/ppsspp/PSP/ISO/Crisis Core - Final Fantasy VII (USA).cso", {st_mode=S_IFREG|0644, st_size=1175296563, ...}) = 0
openat(AT_FDCWD, "/home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so", O_RDONLY|O_CLOEXEC) = 30
read(30, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260C\10\0\0\0\0\0"..., 832) = 832
fstat(30, {st_mode=S_IFREG|0755, st_size=28513544, ...}) = 0
mmap(NULL, 16942752, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 30, 0) = 0x7f4dd84d7000
mprotect(0x7f4dd8f71000, 2097152, PROT_NONE) = 0
mmap(0x7f4dd9171000, 393216, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 30, 0xa9a000) = 0x7f4dd9171000
mmap(0x7f4dd91d1000, 3335840, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f4dd91d1000
close(30)                               = 0
mprotect(0x7f4dd9171000, 360448, PROT_READ) = 0
statfs("/dev/shm/", {f_type=TMPFS_MAGIC, f_bsize=4096, f_blocks=4093799, f_bfree=3979360, f_bavail=3979360, f_files=4093799, f_ffree=4093557, f_fsid={val=[0, 0]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_NOSUID|ST_NODEV}) = 0
futex(0x7f4dffa08370, FUTEX_WAKE_PRIVATE, 2147483647) = 0
openat(AT_FDCWD, "/dev/shm/PPSSPP_ID", O_RDWR|O_CREAT|O_NOFOLLOW|O_CLOEXEC, 000) = -1 EACCES (Permission denied)
ftruncate(-1, 4096)                     = -1 EBADF (Bad file descriptor)
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)

Windows errors (courtesy of another user):

[ERROR] Failed to open libretro core: "Y:\retroarch\cores\ppsspp_libretro.dll"
[ERROR] Error(s): A dynamic link library (DLL) initialization routine failed.

Exception code=0xc0000005 flags=0x0 at 0x00007FF8FD61AD79. Access violation - attempting to write data at address 0x0000000000000000
Exception code=0xc0000005 flags=0x0 at 0x00007FF8FD61BFF0. Access violation - attempting to execute data at address 0x00007FF8FD61BFF0
Exception code=0xc0000005 flags=0x0 at 0x00007FF8FE0EAD79. Access violation - attempting to write data at address 0x0000000000000000
Exception code=0xc0000005 flags=0x0 at 0x00007FF8FE0EAD79. Access violation - attempting to write data at address 0x0000000000000000

[unknownbrackets]

openat(AT_FDCWD, "/dev/shm/PPSSPP_ID", O_RDWR|O_CREAT|O_NOFOLLOW|O_CLOEXEC, 000) = -1 EACCES (Permission denied) ftruncate(-1, 4096) = -1 EBADF (Bad file descriptor)

It seems like it's failing to open the "instance id" thing added for network play, and then potentially trying to close it which crashes. Are you certain dd79d33f147410a91e47bc2c0e4a876ef79c3589 works fine?

-[Unknown]

[hrydgard]

Ah, yeah, should turn off the instance ID stuff in retro builds. Easily done with a couple #ifdef's, I'll do it tonight if noone beats me to it.

[Sanaki]

Are you certain dd79d33 works fine?

Yeah, I just confirmed that commit builds and works fine.

[hrydgard]

Can you get a stack trace, with gdb or something?

[Sanaki]

As requested, using f617bfce:

Thread 1 "retroarch" received signal SIGSEGV, Segmentation fault.
__memset_avx2_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:141
141 ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: No such file or directory.
(gdb) backtrace
#0  0x00007fffed62a19d in __memset_avx2_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:141
#1  0x00007fffb6266ec7 in AudioChannel::clear() () at /home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so
#2  0x00007fffb5f36eda in _GLOBAL__sub_I_sceAudio.cpp () at /home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so
#3  0x00007ffff7de5783 in call_init (env=0x7fffffffdb98, argv=0x7fffffffdb88, argc=1, l=<optimized out>) at dl-init.c:72
#4  0x00007ffff7de5783 in _dl_init (main_map=main_map@entry=0x5555578c98e0, argc=1, argv=0x7fffffffdb88, env=0x7fffffffdb98) at dl-init.c:119
#5  0x00007ffff7dea24f in dl_open_worker (a=a@entry=0x7fffffff27f0) at dl-open.c:522
#6  0x00007fffed60251f in __GI__dl_catch_exception (exception=0x7fffffff27d0, operate=0x7ffff7de9e10 <dl_open_worker>, args=0x7fffffff27f0) at dl-error-skeleton.c:196
#7  0x00007ffff7de981a in _dl_open
    (file=0x55555664d537 <rarch_st+98135> "/home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so", mode=-2147483647, caller_dlopen=0x5555556ab7cc <dylib_load+29>, nsid=<optimized out>, argc=1, argv=<optimized out>, env=0x7fffffffdb98) at dl-open.c:605
#8  0x00007fffee218f96 in dlopen_doit (a=a@entry=0x7fffffff2a20) at dlopen.c:66
#9  0x00007fffed60251f in __GI__dl_catch_exception (exception=exception@entry=0x7fffffff29c0, operate=0x7fffee218f40 <dlopen_doit>, args=0x7fffffff2a20) at dl-error-skeleton.c:196
#10 0x00007fffed6025af in __GI__dl_catch_error (objname=0x5555567b87f0, errstring=0x5555567b87f8, mallocedp=0x5555567b87e8, operate=<optimized out>, args=<optimized out>)
    at dl-error-skeleton.c:215
#11 0x00007fffee219745 in _dlerror_run (operate=operate@entry=0x7fffee218f40 <dlopen_doit>, args=args@entry=0x7fffffff2a20) at dlerror.c:162
#12 0x00007fffee219051 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#13 0x00005555556ab7cc in dylib_load (path=0x55555664d537 <rarch_st+98135> "/home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so") at libretro-common/dynamic/dylib.c:123
#14 0x0000555555622f1e in libretro_get_system_info_lib
    (path=0x55555664d537 <rarch_st+98135> "/home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so", info=0x7fffffff2b10, load_no_content=0x555556682400 <rarch_st+314912>)
    at retroarch.c:20179
#15 0x0000555555623049 in libretro_get_system_info
    (p_rarch=0x5555566355e0 <rarch_st>, path=0x55555664d537 <rarch_st+98135> "/home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so", info=0x5555566823d8 <rarch_st+314872>, load_no_content=0x555556682400 <rarch_st+314912>) at retroarch.c:20245
#16 0x0000555555619f63 in command_event (cmd=CMD_EVENT_LOAD_CORE_PERSIST, data=0x0) at retroarch.c:15784
#17 0x0000555555619fdf in command_event (cmd=CMD_EVENT_LOAD_CORE, data=0x0) at retroarch.c:15805
#18 0x00005555556669d5 in task_push_load_content_from_playlist_from_menu
    (core_path=0x7fffffffb520 "/home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so", fullpath=0x7fffffff9520 "/home/ccrowley/.config/ppsspp/PSP/ISO/Crisis Core - Final Fantasy VII (USA).cso", label=0x7fffffffa520 "Crisis Core - Final Fantasy VII (USA)", content_info=0x7fffffff73b0, cb=0x0, user_data=0x0) at tasks/task_content.c:1717
#19 0x00005555559394ce in default_action_ok_load_content_from_playlist_from_menu
    (_path=0x7fffffffb520 "/home/ccrowley/.config/retroarch/cores/ppsspp_libretro.so", path=0x7fffffff9520 "/home/ccrowley/.config/ppsspp/PSP/ISO/Crisis Core - Final Fantasy VII (USA).cso", entry_label=0x7fffffffa520 "Crisis Core - Final Fantasy VII (USA)") at menu/cbs/menu_cbs_ok.c:1897
#20 0x000055555593a665 in action_ok_playlist_entry_collection (path=0x7fffffffcbf8 "Run", label=0x7fffffffccf7 "collection", type=7, idx=0, entry_idx=0) at menu/cbs/menu_cbs_ok.c:2271
#21 0x0000555555602286 in generic_menu_entry_action (userdata=0x5555573b2080, entry=0x7fffffffcbe0, i=0, action=MENU_ACTION_OK) at retroarch.c:4704
#22 0x00005555558dd004 in xmb_menu_entry_action (userdata=0x5555573b2080, entry=0x7fffffffcbe0, i=0, action=MENU_ACTION_OK) at menu/drivers/xmb.c:7067
#23 0x00005555556040bf in menu_entry_action (entry=0x7fffffffcbe0, i=0, action=MENU_ACTION_OK) at retroarch.c:5405
#24 0x0000555555601e6f in generic_menu_iterate (p_rarch=0x5555566355e0 <rarch_st>, data=0x5555573aa030, userdata=0x5555573b2080, action=MENU_ACTION_OK, current_time=1335571430651)
    at retroarch.c:4626
#25 0x00005555556070b8 in menu_driver_iterate (iterate=0x7fffffffd790, current_time=1335571430651) at retroarch.c:7006
#26 0x000055555564c8f5 in runloop_check_state (p_rarch=0x5555566355e0 <rarch_st>, settings=0x7ffff7fbc010, current_time=1335571430651) at retroarch.c:38887
#27 0x000055555564e376 in runloop_iterate () at retroarch.c:39479
#28 0x000055555561d463 in rarch_main (argc=1, argv=0x7fffffffdb88, data=0x0) at retroarch.c:17494
#29 0x00005555557ca46b in main(int, char**) (argc=1, argv=0x7fffffffdb88) at ui/drivers/qt/ui_qt_application.cpp:151

[hrydgard]

That's one weird crash. AudioChannel::clear() is very straightforward. Feels like we have some memory corruption, maybe of "index"... But when? This is while we are running the global constructors for the PPSSPP retroarch library. Wacky.

[hrydgard]

Did an experimental commit to master, but I kinda hope it won't help because if it does, I really don't understand what's going on...

[unknownbrackets]

Maybe some compiler bug in the order of constructors and chanSampleQueues not ready yet? Strange...

-[Unknown]

[bslenul]

Did an experimental commit to master, but I kinda hope it won't help because if it does, I really don't understand what's going on...

Well, bad news then I guess? :p 27c059bf9ed350772022a6571ba10ce5d89c01a0 fixes it on Windows at least, games are loading fine again with RetroArch. I'll let Sanaki confirm or not on Linux side.

[Sanaki]

Yup, fixed here too. Leaving the issue open for the moment in case you want to address it further here, but feel free to close it if not.

[hrydgard]

It is a case of constructors getting executed in an undefined order. Surprised we haven't run into this before...