search

philippe44 / AirConnect

Use AirPlay to stream to UPnP/Sonos & Chromecast devices
Other
3.46k stars 217 forks source link

Segmentation fault #308

Closed dbogdanov closed 3 years ago

dbogdanov commented 3 years ago

Running v0.2.30.0 on RPi4, Ubuntu Server 20.10 (aarch64), I am getting occasional segfaults:

[14:31:25.878481] handle_rtsp:398 [0xffff8000bae0]: received FLUSH
[14:31:25.878896] hairtunes_flush:503 [0xffff84006360]: flush 33350 2934727424
[14:31:25.878983] callback:197 [0x33ef06b0]: Flush
[14:31:25.879249] CastStop:245 [0x33ef06b0]: Immediate STOP (id:458)
[14:31:25.879563] handle_rtsp:604 [0xffff8000bae0]: responding:
RTSP/1.0 200 OK
Audio-Jack-Status: connected; type=analog
CSeq: 10

[14:31:25.944741] http_thread_func:1096 HTTP close 16
[14:36:26.008316] MRThread:332 [0x33ef06b0]: Cast peer closed connection
[14:55:36.962821] handle_rtsp:398 [0xffff8000bae0]: received TEARDOWN
[14:55:36.986812] rtp_thread_func:842 [0xffff84006360]: terminating
[14:55:36.986896] http_thread_func:1233 [0xffff84006360]: terminating
Segmentation fault (core dumped)
philippe44 commented 3 years ago

Would you know how to run it under gdb?

dbogdanov commented 3 years ago 
[12:48:26.198238] handle_rtsp:398 [0xffffe800be70]: received TEARDOWN
[12:48:26.207311] rtp_thread_func:842 [0xffffec2e9320]: terminating
[Thread 0xffffbffff150 (LWP 19033) exited]
[12:48:26.226657] http_thread_func:1233 [0xffffec2e9320]: terminating
[Thread 0xffffbf7fe150 (LWP 19034) exited]

Thread 19 "aircast-aarch64" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffffe4df8150 (LWP 19032)]
tcache_thread_shutdown () at malloc.c:2983
2983    malloc.c: No such file or directory.
(gdb)

(gdb) backtrace
#0  tcache_thread_shutdown () at malloc.c:2983
#1  __malloc_arena_thread_freeres () at arena.c:951
#2  0x0000fffff7dca784 in __libc_thread_freeres () at thread-freeres.c:42
#3  0x0000fffff7fa1f88 in start_thread (arg=0x0) at pthread_create.c:477
#4  0x0000fffff7e203dc in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

I haven't looked at building from source with debug symbols.

philippe44 commented 3 years ago

Thanks but I need to provide you with a version w/ symbols otherwise it will be difficult for me to figure out. here it is https://www.dropbox.com/s/zobszl09w2otpg0/aircast-aarch64?dl=0. For aarch64, non-static version with all debug symbols left

meiser79 commented 3 years ago

I see a similar crash with aircast 0.2.42.0 on Ubuntu 20.04 x86_64. EDIT: The same happens with airupnp 02.42.0.

[09:48:31.975884] ProcessQueue:583 [0xdcf730]: Processing VOLUME (id:9)
[09:48:32.010414] buffer_put_packet:643 [0x7f61ec00ae30]: fill [level:3] [W:5806 R:5804]
[09:48:32.023080] _buffer_get_frame:983 [0x7f61ec00ae30]: drain [level:4 gap:982] [W:5810 R:5806] [R:0 S:0 F:0]
[09:48:32.269348] MRThread:302 [0xdcf730]: Cast stopped by other remote
[09:48:32.284358] http_thread_func:1101 HTTP close 29
[09:48:35.659780] buffer_put_packet:643 [0x7f6204006410]: fill [level:2] [W:6265 R:6264]
[09:48:35.665249] _buffer_get_frame:983 [0x7f6204006410]: drain [level:1 gap:1000] [W:6266 R:6265] [R:0 S:0 F:0]
[09:48:36.086121] buffer_put_packet:643 [0x7f61ec00ae30]: fill [level:477] [W:6318 R:5842]
[09:48:39.250503] handle_rtsp:397 [0x7f620c00ebf0]: received TEARDOWN
[09:48:39.260989] http_thread_func:1238 [0x7f6204006410]: terminating
[09:48:39.272711] rtp_thread_func:848 [0x7f6204006410]: terminating
Segmentation fault`

This happens when I disconnect from a Google Nest Mini. If you provided a binary with symbols, I could attach the gdb backtrace.

Regards, meiser

philippe44 commented 3 years ago

https://www.dropbox.com/s/5aciiphd0d67054/aircast-x86-64?dl=0

meiser79 commented 3 years ago

Even with the binary with included symbols, the backtrace looks the same as above:


Thread 17 "aircast-x86-64" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe77fe700 (LWP 662703)]
tcache_thread_shutdown () at malloc.c:2963
2963    malloc.c: No such file or directory.
(gdb) bt
#0  tcache_thread_shutdown () at malloc.c:2963
#1  __malloc_arena_thread_freeres () at arena.c:951
#2  0x00007ffff7ced220 in __libc_thread_freeres () at thread-freeres.c:38
#3  0x00007ffff7fa662f in start_thread (arg=<optimized out>) at pthread_create.c:491
#4  0x00007ffff7d6d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Interesting enough, if I run it as MALLOC_CHECK_=1 ./aircast-x86-64 -l 1000:2000 -Z, it doesn't crash.

dbogdanov commented 3 years ago

Similar case here on aarch64, running with debug symbols:

[16:37:29.513687] handle_rtsp:398 [0xffffe800baa0]: received TEARDOWN
[16:37:29.522604] rtp_thread_func:842 [0xffffec006360]: terminating
[Thread 0xffffe77fe150 (LWP 65460) exited]
[16:37:29.527922] http_thread_func:1233 [0xffffec006360]: terminating
[Thread 0xffffe6ffd150 (LWP 65461) exited]

Thread 9 "aircast-aarch64" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffffe7fff150 (LWP 65459)]
tcache_thread_shutdown () at malloc.c:2983
2983    malloc.c: No such file or directory.
(gdb) backtrace
#0  tcache_thread_shutdown () at malloc.c:2983
#1  __malloc_arena_thread_freeres () at arena.c:951
#2  0x0000fffff7dca784 in __libc_thread_freeres () at thread-freeres.c:42
#3  0x0000fffff7fa1f88 in start_thread (arg=0x0) at pthread_create.c:477
#4  0x0000fffff7e203dc in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78
philippe44 commented 3 years ago

Could you use -d raop=sdebug and post logs?

meiser79 commented 3 years ago

Here you are.

log-debug.txt

meiser79 commented 3 years ago

Any news?

philippe44 commented 3 years ago

I have a better idea, but not yet close enough. Can you try the same command (with sdebug), but under gdb and then when it crashed do a backtrace. To add args to a gdb, use --args ./aircast-x86 . Please use attached version that contains debug symbols aircast-x86.zip

meiser79 commented 3 years ago

As I have a x86_64-only system, I used the previously provided version with debug symbols.

Here's the log, but there's no major difference. gdb-log.txt

philippe44 commented 3 years ago

ok strange there is no more backtrace but here is a debug version with ton of logs around where the crash happens aircast-x86-64.zip

meiser79 commented 3 years ago

I just tried again with release version 0.2.43.1. I can't get it to crash. The debug version 0.2.44 also doesn't crash.

@dbogdanov could you check aarch64 with 0.2.43.1?

philippe44 commented 3 years ago

I've corrected a few things that valgrind was complaining about in recent versions (should not have caused a crash but I did not analyze in lots of detail if not of them was potential cause of crash)

philippe44 commented 3 years ago

Can we close this one if you don't observe it anymore? We can always re-open it later

meiser79 commented 3 years ago

Yes, I guess so.