karlheyes
commented
5 years ago can you make debug on that and retry, so that a stack trace can be attained. Maybe run in valgrind and send me the output.
karl.
Closed mylselgan closed 4 years ago
karlheyes
commented
5 years ago can you make debug on that and retry, so that a stack trace can be attained. Maybe run in valgrind and send me the output.
karl.
mylselgan
commented
5 years ago I am not expert but i have managed to install valgrind and run it like following Note: this is not debug build, I run this with same build (Icecast 2.4.0-kh12-20190416123413)
If I need to run with debug build, please guide me following Do i need to remove the existing build? if yes how? or i can simply run ./configure make debug and then make install on top of existing build?
$ **valgrind --leak-check=full -v icecast -c icecast/ice_auth.xml &**
[1] 6594
radio@up231:~$ ==6594== Memcheck, a memory error detector
==6594== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==6594== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==6594== Command: icecast -c icecast/ice_auth.xml
==6594==
--6594-- Valgrind options:
--6594-- --leak-check=full
--6594-- -v
--6594-- Contents of /proc/version:
--6594-- Linux version 4.9.0-8-amd64 (debian-kernel@lists.debian.org) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) ) #1 SMP Debian 4.9.110-3+deb9u4 (2018-08-21)
--6594--
--6594-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-rdtscp-sse3
--6594-- Page sizes: currently 4096, max supported 4096
--6594-- Valgrind library directory: /usr/lib/valgrind
--6594-- Reading syms from /usr/local/bin/icecast
--6594-- Reading syms from /lib/x86_64-linux-gnu/ld-2.24.so
--6594-- Considering /usr/lib/debug/.build-id/60/6df9c355103e82140d513bc7a25a635591c153.debug ..
--6594-- .. build-id is valid
--6594-- Reading syms from /usr/lib/valgrind/memcheck-amd64-linux
--6594-- Considering /usr/lib/valgrind/memcheck-amd64-linux ..
--6594-- .. CRC mismatch (computed db5b2ec5 wanted 0eae776b)
--6594-- Considering /usr/lib/debug/usr/lib/valgrind/memcheck-amd64-linux ..
--6594-- .. CRC is valid
--6594-- object doesn't have a dynamic symbol table
--6594-- Scheduler: using generic scheduler lock implementation.
--6594-- Reading suppressions file: /usr/lib/valgrind/default.supp
==6594== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-6594-by-radio-on-???
==6594== embedded gdbserver: writing to /tmp/vgdb-pipe-to-vgdb-from-6594-by-radio-on-???
==6594== embedded gdbserver: shared mem /tmp/vgdb-pipe-shared-mem-vgdb-6594-by-radio-on-???
==6594==
==6594== TO CONTROL THIS PROCESS USING vgdb (which you probably
==6594== don't want to do, unless you know exactly what you're doing,
==6594== or are doing some strange experiment):
==6594== /usr/lib/valgrind/../../bin/vgdb --pid=6594 ...command...
==6594==
==6594== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==6594== /path/to/gdb icecast
==6594== and then give GDB the following command
==6594== target remote | /usr/lib/valgrind/../../bin/vgdb --pid=6594
==6594== --pid is optional if only one valgrind process is running
==6594==
--6594-- REDIR: 0x401aec0 (ld-linux-x86-64.so.2:strlen) redirected to 0x3809de81 (vgPlain_amd64_linux_REDIR_FOR_strlen)
--6594-- REDIR: 0x4019770 (ld-linux-x86-64.so.2:index) redirected to 0x3809de9b (vgPlain_amd64_linux_REDIR_FOR_index)
--6594-- Reading syms from /usr/lib/valgrind/vgpreload_core-amd64-linux.so
--6594-- Considering /usr/lib/valgrind/vgpreload_core-amd64-linux.so ..
--6594-- .. CRC mismatch (computed 74a069fa wanted 84d99202)
--6594-- Considering /usr/lib/debug/usr/lib/valgrind/vgpreload_core-amd64-linux.so ..
--6594-- .. CRC is valid
--6594-- Reading syms from /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
--6594-- Considering /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so ..
--6594-- .. CRC mismatch (computed 88f2547e wanted 8a7a4459)
--6594-- Considering /usr/lib/debug/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so ..
--6594-- .. CRC is valid
==6594== WARNING: new redirection conflicts with existing -- ignoring it
--6594-- old: 0x0401aec0 (strlen ) R-> (0000.0) 0x3809de81 vgPlain_amd64_linux_REDIR_FOR_strlen
--6594-- new: 0x0401aec0 (strlen ) R-> (2007.0) 0x04c2ee60 strlen
--6594-- REDIR: 0x4019990 (ld-linux-x86-64.so.2:strcmp) redirected to 0x4c2ff60 (strcmp)
--6594-- REDIR: 0x401b9d0 (ld-linux-x86-64.so.2:mempcpy) redirected to 0x4c33330 (mempcpy)
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.4.0
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.8
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libogg.so.0.8.2
--6594-- Considering /usr/lib/x86_64-linux-gnu/libogg.so.0.8.2 ..
--6594-- .. CRC mismatch (computed ac481d9c wanted 6c4c8029)
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libm-2.24.so
--6594-- Considering /usr/lib/debug/.build-id/4e/49714c557ce0472c798f39365ca10f9c0e1933.debug ..
--6594-- .. build-id is valid
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libxslt.so.1.1.29
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.4
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libpthread-2.24.so
--6594-- Considering /usr/lib/debug/.build-id/16/d609487bcc4acbac29a4eaa2dda0d2f56211ec.debug ..
--6594-- .. build-id is valid
--6594-- Reading syms from /lib/x86_64-linux-gnu/libc-2.24.so
--6594-- Considering /usr/lib/debug/.build-id/77/5143e680ff0cd4cd51cce1ce8ca216e635a1d6.debug ..
--6594-- .. build-id is valid
--6594-- Reading syms from /lib/x86_64-linux-gnu/libdl-2.24.so
--6594-- Considering /usr/lib/debug/.build-id/db/2caeeec37482a98ab1416d0a9afe2944930de9.debug ..
--6594-- .. build-id is valid
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.12.3
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libidn2.so.0.1.4
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/librtmp.so.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libpsl.so.5.1.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libnettle.so.6.3
--6594-- Considering /usr/lib/x86_64-linux-gnu/libnettle.so.6.3 ..
--6594-- .. CRC mismatch (computed 51e967a4 wanted 165ded96)
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libgnutls.so.30.13.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libcom_err.so.2.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.7
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.7
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libz.so.1.2.8
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libicui18n.so.57.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libicuuc.so.57.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libicudata.so.57.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/liblzma.so.5.2.2
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libunistring.so.0.1.2
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libhogweed.so.4.3
--6594-- Considering /usr/lib/x86_64-linux-gnu/libhogweed.so.4.3 ..
--6594-- .. CRC mismatch (computed 64c89d37 wanted 085749a6)
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.2
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.2.0
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libidn.so.11.6.16
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.3
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libkeyutils.so.1.5
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libresolv-2.24.so
--6594-- Considering /usr/lib/debug/.build-id/ea/d5fd817712e63c1212d1ee7d7ee1b9c29f93a7.debug ..
--6594-- .. build-id is valid
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libgcc_s.so.1
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
--6594-- object doesn't have a symbol table
--6594-- Reading syms from /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
--6594-- object doesn't have a symbol table
--6594-- REDIR: 0x6801710 (libc.so.6:strcasecmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67fd1b0 (libc.so.6:strcspn) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x6803a00 (libc.so.6:strncasecmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67ff620 (libc.so.6:strpbrk) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67ff9b0 (libc.so.6:strspn) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x6800d80 (libc.so.6:memmove) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67fd090 (libc.so.6:strcpy) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67fb9b0 (libc.so.6:index) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x6801230 (libc.so.6:memset) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x6806100 (libc.so.6:memcpy@@GLIBC_2.14) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67ff2f0 (libc.so.6:strncpy) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67fda50 (libc.so.6:strncmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67fda10 (libc.so.6:strncat) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x6800940 (libc.so.6:bcmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67fbc00 (libc.so.6:strcmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x68734e0 (libc.so.6:__memcpy_chk) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x67fb7b0 (libc.so.6:strcat) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x6801560 (libc.so.6:stpcpy) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x68003c0 (libc.so.6:strstr) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--6594-- REDIR: 0x68735b0 (libc.so.6:__memmove_chk) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
==6594== WARNING: new redirection conflicts with existing -- ignoring it
--6594-- old: 0x06800e80 (__memcpy_chk_sse2_un) R-> (2030.0) 0x04c33420 __memcpy_chk
--6594-- new: 0x06800e80 (__memcpy_chk_sse2_un) R-> (2024.0) 0x04c32df0 __memmove_chk
==6594== WARNING: new redirection conflicts with existing -- ignoring it
--6594-- old: 0x06800e80 (__memcpy_chk_sse2_un) R-> (2030.0) 0x04c33420 __memcpy_chk
--6594-- new: 0x06800e80 (__memcpy_chk_sse2_un) R-> (2024.0) 0x04c32df0 __memmove_chk
--6594-- REDIR: 0x67ff330 (libc.so.6:rindex) redirected to 0x4c2e7f0 (rindex)
--6594-- REDIR: 0x67fd650 (libc.so.6:strlen) redirected to 0x4c2eda0 (strlen)
--6594-- REDIR: 0x67f7f10 (libc.so.6:malloc) redirected to 0x4c2bb40 (malloc)
--6594-- REDIR: 0x6800e90 (libc.so.6:memcpy@GLIBC_2.2.5) redirected to 0x4c300c0 (memcpy@GLIBC_2.2.5)
--6594-- REDIR: 0x67fbc40 (libc.so.6:__GI_strcmp) redirected to 0x4c2fe70 (__GI_strcmp)
--6594-- REDIR: 0x67f8900 (libc.so.6:calloc) redirected to 0x4c2db10 (calloc)
--6594-- REDIR: 0x67fb9e0 (libc.so.6:__GI_strchr) redirected to 0x4c2e950 (__GI_strchr)
--6594-- REDIR: 0x6800e80 (libc.so.6:__memcpy_chk_sse2_unaligned) redirected to 0x4c33420 (__memcpy_chk)
--6594-- REDIR: 0x680c520 (libc.so.6:__strcmp_sse2_unaligned) redirected to 0x4c2fe20 (strcmp)
--6594-- REDIR: 0x67f8510 (libc.so.6:free) redirected to 0x4c2cd70 (free)
--6594-- REDIR: 0x68c0f80 (libc.so.6:__memcmp_sse4_1) redirected to 0x4c31ca0 (__memcmp_sse4_1)
--6594-- REDIR: 0x67f85c0 (libc.so.6:realloc) redirected to 0x4c2dd50 (realloc)
--6594-- REDIR: 0x67fdaa0 (libc.so.6:__GI_strncmp) redirected to 0x4c2f550 (__GI_strncmp)
--6594-- REDIR: 0x67fff50 (libc.so.6:__GI_strstr) redirected to 0x4c33590 (__strstr_sse2)
--6594-- REDIR: 0x68005f0 (libc.so.6:memchr) redirected to 0x4c30000 (memchr)
--6594-- REDIR: 0x680c4d0 (libc.so.6:__GI_strncpy) redirected to 0x4c2f150 (__GI_strncpy)
--6594-- REDIR: 0x68a2d20 (libc.so.6:__strpbrk_sse42) redirected to 0x4c33670 (strpbrk)
--6594-- REDIR: 0x6807710 (libc.so.6:strchrnul) redirected to 0x4c32e60 (strchrnul)
--6594-- REDIR: 0x6800e70 (libc.so.6:__GI_mempcpy) redirected to 0x4c33060 (__GI_mempcpy)
--6594-- REDIR: 0x68015a0 (libc.so.6:__GI_stpcpy) redirected to 0x4c31f80 (__GI_stpcpy)
--6594-- Reading syms from /lib/x86_64-linux-gnu/libnss_files-2.24.so
--6594-- Considering /usr/lib/debug/.build-id/8a/db8c8cdde0e4c3c3babb8874ae69dca15a666a.debug ..
--6594-- .. build-id is valid
--6594-- REDIR: 0x67fd0d0 (libc.so.6:__GI_strcpy) redirected to 0x4c2ef50 (__GI_strcpy)
--6594-- REDIR: 0x6801300 (libc.so.6:__GI_memset) redirected to 0x4c324c0 (memset)
--6594-- REDIR: 0x689f060 (libc.so.6:__strcasecmp_sse42) redirected to 0x4c2f6a0 (strcasecmp)
--6594-- REDIR: 0x6811170 (libc.so.6:__strcpy_sse2_unaligned) redirected to 0x4c2ee80 (strcpy)
--6594-- REDIR: 0x68a0c10 (libc.so.6:__strncasecmp_sse42) redirected to 0x4c2f780 (strncasecmp)
--6594-- REDIR: 0x6800980 (libc.so.6:__GI_memcmp) redirected to 0x4c31b40 (__GI_memcmp)
--6594-- REDIR: 0x6807500 (libc.so.6:rawmemchr) redirected to 0x4c32e90 (rawmemchr)
--6594-- REDIR: 0x68117a0 (libc.so.6:__strncpy_sse2_unaligned) redirected to 0x4c2f3b0 (__strncpy_sse2_unaligned)
--6594-- REDIR: 0x689e080 (libc.so.6:__strncmp_sse42) redirected to 0x4c2f630 (__strncmp_sse42)
--6594-- REDIR: 0x68a2be0 (libc.so.6:__strcspn_sse42) redirected to 0x4c336d0 (strcspn)
failed to increase max number of open files from 1024 to 5000
--6594-- REDIR: 0x6815810 (libc.so.6:__strstr_sse2_unaligned) redirected to 0x4c33520 (strstr)
--6594-- Reading syms from /lib/x86_64-linux-gnu/libnss_dns-2.24.so
--6594-- Considering /usr/lib/debug/.build-id/b6/5aa56f4cecc441fce7f85f0b4aa0f2815e4e34.debug ..
--6594-- .. build-id is valid
--6594-- REDIR: 0x68a2e50 (libc.so.6:__strspn_sse42) redirected to 0x4c33730 (strspn)
--6594-- REDIR: 0x67fd7f0 (libc.so.6:strnlen) redirected to 0x4c2ed40 (strnlen)
==6594== Thread 4:
==6594== Invalid read of size 4
==6594== at 0x11E881: source_listener_detach (source.c:1139)
==6594== by 0x121F29: source_listener_release (source.c:2339)
==6594== by 0x121F29: send_to_listener (source.c:1231)
==6594== by 0x1279A4: worker (client.c:735)
==6594== by 0x143714: _start_routine (thread.c:727)
==6594== by 0x65674A3: start_thread (pthread_create.c:456)
==6594== by 0x6865D0E: clone (clone.S:97)
==6594== Address 0x10af72a0 is 0 bytes inside a block of size 40 free'd
==6594== at 0x4C2CDDB: free (vg_replace_malloc.c:530)
==6594== by 0x11FA50: source_read (source.c:674)
==6594== by 0x119847: relay_read (slave.c:1423)
==6594== by 0x1279A4: worker (client.c:735)
==6594== by 0x143714: _start_routine (thread.c:727)
==6594== by 0x65674A3: start_thread (pthread_create.c:456)
==6594== by 0x6865D0E: clone (clone.S:97)
==6594== Block was alloc'd at
==6594== at 0x4C2DBC5: calloc (vg_replace_malloc.c:711)
==6594== by 0x1265D6: refbuf_new (refbuf.c:69)
==6594== by 0x13227D: validate_mpeg (format_mp3.c:936)
==6594== by 0x1340F8: mp3_get_no_meta (format_mp3.c:961)
==6594== by 0x11F8B0: source_read (source.c:575)
==6594== by 0x119847: relay_read (slave.c:1423)
==6594== by 0x1279A4: worker (client.c:735)
==6594== by 0x143714: _start_routine (thread.c:727)
==6594== by 0x65674A3: start_thread (pthread_create.c:456)
==6594== by 0x6865D0E: clone (clone.S:97)
==6594==The stream is still up and not crashed yet
karlheyes
commented
5 years ago ok, I think I know what that is down to. I've done soem tuning to timing to avoid the case of queue purging poast a point where listeners are still at. The fault above can cause a problem if the memory is reused before that routine, in the above case, it wasn't so hasn't crashed. Lagging clients seem to be the trigger. I've committed a fix to git.
karl.
mylselgan
commented
5 years ago Hi, Crashed again, Total connection to all mounts will be 300 to 500 Updated to the master branch,
wget https://github.com/karlheyes/icecast-kh/archive/master.zip
unzip master.zip
cd icecast-kh-master
./configure
sudo make
sudo install
icecast -v
**Icecast 2.4.0-kh12-20190417110839**[1]+ Aborted (core dumped) icecast -c icecast/ice_auth.xml
$ valgrind --leak-check=full -v icecast -c icecast/ice_auth.xml &
[2] 18183
==18183== Memcheck, a memory error detector
==18183== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==18183== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==18183== Command: icecast -c icecast/ice_auth.xml
==18183==
--18183-- Valgrind options:
--18183-- --leak-check=full
--18183-- -v
--18183-- Contents of /proc/version:
--18183-- Linux version 4.9.0-8-amd64 (debian-kernel@lists.debian.org) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) ) #1 SMP Debian 4.9.110-3+deb9u4 (2018-08-21)
--18183--
--18183-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-rdtscp-sse3
--18183-- Page sizes: currently 4096, max supported 4096
--18183-- Valgrind library directory: /usr/lib/valgrind
--18183-- Reading syms from /usr/local/bin/icecast
--18183-- Reading syms from /lib/x86_64-linux-gnu/ld-2.24.so
--18183-- Considering /usr/lib/debug/.build-id/60/6df9c355103e82140d513bc7a25a635591c153.debug ..
--18183-- .. build-id is valid
--18183-- Reading syms from /usr/lib/valgrind/memcheck-amd64-linux
--18183-- Considering /usr/lib/valgrind/memcheck-amd64-linux ..
--18183-- .. CRC mismatch (computed db5b2ec5 wanted 0eae776b)
--18183-- Considering /usr/lib/debug/usr/lib/valgrind/memcheck-amd64-linux ..
--18183-- .. CRC is valid
--18183-- object doesn't have a dynamic symbol table
--18183-- Scheduler: using generic scheduler lock implementation.
--18183-- Reading suppressions file: /usr/lib/valgrind/default.supp
==18183== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-18183-by-radio-on-???
==18183== embedded gdbserver: writing to /tmp/vgdb-pipe-to-vgdb-from-18183-by-radio-on-???
==18183== embedded gdbserver: shared mem /tmp/vgdb-pipe-shared-mem-vgdb-18183-by-radio-on-???
==18183==
==18183== TO CONTROL THIS PROCESS USING vgdb (which you probably
==18183== don't want to do, unless you know exactly what you're doing,
==18183== or are doing some strange experiment):
==18183== /usr/lib/valgrind/../../bin/vgdb --pid=18183 ...command...
==18183==
==18183== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==18183== /path/to/gdb icecast
==18183== and then give GDB the following command
==18183== target remote | /usr/lib/valgrind/../../bin/vgdb --pid=18183
==18183== --pid is optional if only one valgrind process is running
==18183==
--18183-- REDIR: 0x401aec0 (ld-linux-x86-64.so.2:strlen) redirected to 0x3809de81 (vgPlain_amd64_linux_REDIR_FOR_strlen)
--18183-- REDIR: 0x4019770 (ld-linux-x86-64.so.2:index) redirected to 0x3809de9b (vgPlain_amd64_linux_REDIR_FOR_index)
--18183-- Reading syms from /usr/lib/valgrind/vgpreload_core-amd64-linux.so
--18183-- Considering /usr/lib/valgrind/vgpreload_core-amd64-linux.so ..
--18183-- .. CRC mismatch (computed 74a069fa wanted 84d99202)
--18183-- Considering /usr/lib/debug/usr/lib/valgrind/vgpreload_core-amd64-linux.so ..
--18183-- .. CRC is valid
--18183-- Reading syms from /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
--18183-- Considering /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so ..
--18183-- .. CRC mismatch (computed 88f2547e wanted 8a7a4459)
--18183-- Considering /usr/lib/debug/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so ..
--18183-- .. CRC is valid
==18183== WARNING: new redirection conflicts with existing -- ignoring it
--18183-- old: 0x0401aec0 (strlen ) R-> (0000.0) 0x3809de81 vgPlain_amd64_linux_REDIR_FOR_strlen
--18183-- new: 0x0401aec0 (strlen ) R-> (2007.0) 0x04c2ee60 strlen
--18183-- REDIR: 0x4019990 (ld-linux-x86-64.so.2:strcmp) redirected to 0x4c2ff60 (strcmp)
--18183-- REDIR: 0x401b9d0 (ld-linux-x86-64.so.2:mempcpy) redirected to 0x4c33330 (mempcpy)
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.4.0
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.8
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libogg.so.0.8.2
--18183-- Considering /usr/lib/x86_64-linux-gnu/libogg.so.0.8.2 ..
--18183-- .. CRC mismatch (computed ac481d9c wanted 6c4c8029)
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libm-2.24.so
--18183-- Considering /usr/lib/debug/.build-id/4e/49714c557ce0472c798f39365ca10f9c0e1933.debug ..
--18183-- .. build-id is valid
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libxslt.so.1.1.29
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.4
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libpthread-2.24.so
--18183-- Considering /usr/lib/debug/.build-id/16/d609487bcc4acbac29a4eaa2dda0d2f56211ec.debug ..
--18183-- .. build-id is valid
--18183-- Reading syms from /lib/x86_64-linux-gnu/libc-2.24.so
--18183-- Considering /usr/lib/debug/.build-id/77/5143e680ff0cd4cd51cce1ce8ca216e635a1d6.debug ..
--18183-- .. build-id is valid
--18183-- Reading syms from /lib/x86_64-linux-gnu/libdl-2.24.so
--18183-- Considering /usr/lib/debug/.build-id/db/2caeeec37482a98ab1416d0a9afe2944930de9.debug ..
--18183-- .. build-id is valid
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.12.3
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libidn2.so.0.1.4
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/librtmp.so.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libpsl.so.5.1.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libnettle.so.6.3
--18183-- Considering /usr/lib/x86_64-linux-gnu/libnettle.so.6.3 ..
--18183-- .. CRC mismatch (computed 51e967a4 wanted 165ded96)
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libgnutls.so.30.13.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libcom_err.so.2.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.7
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.7
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libz.so.1.2.8
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libicui18n.so.57.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libicuuc.so.57.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libicudata.so.57.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/liblzma.so.5.2.2
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libunistring.so.0.1.2
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libhogweed.so.4.3
--18183-- Considering /usr/lib/x86_64-linux-gnu/libhogweed.so.4.3 ..
--18183-- .. CRC mismatch (computed 64c89d37 wanted 085749a6)
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.2
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.2.0
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libidn.so.11.6.16
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.3
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libkeyutils.so.1.5
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libresolv-2.24.so
--18183-- Considering /usr/lib/debug/.build-id/ea/d5fd817712e63c1212d1ee7d7ee1b9c29f93a7.debug ..
--18183-- .. build-id is valid
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libgcc_s.so.1
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
--18183-- object doesn't have a symbol table
--18183-- Reading syms from /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
--18183-- object doesn't have a symbol table
--18183-- REDIR: 0x6801710 (libc.so.6:strcasecmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67fd1b0 (libc.so.6:strcspn) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x6803a00 (libc.so.6:strncasecmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67ff620 (libc.so.6:strpbrk) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67ff9b0 (libc.so.6:strspn) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x6800d80 (libc.so.6:memmove) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67fd090 (libc.so.6:strcpy) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67fb9b0 (libc.so.6:index) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x6801230 (libc.so.6:memset) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x6806100 (libc.so.6:memcpy@@GLIBC_2.14) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67ff2f0 (libc.so.6:strncpy) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67fda50 (libc.so.6:strncmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67fda10 (libc.so.6:strncat) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x6800940 (libc.so.6:bcmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67fbc00 (libc.so.6:strcmp) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x68734e0 (libc.so.6:__memcpy_chk) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x67fb7b0 (libc.so.6:strcat) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x6801560 (libc.so.6:stpcpy) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x68003c0 (libc.so.6:strstr) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
--18183-- REDIR: 0x68735b0 (libc.so.6:__memmove_chk) redirected to 0x4a26740 (_vgnU_ifunc_wrapper)
==18183== WARNING: new redirection conflicts with existing -- ignoring it
--18183-- old: 0x06800e80 (__memcpy_chk_sse2_un) R-> (2030.0) 0x04c33420 __memcpy_chk
--18183-- new: 0x06800e80 (__memcpy_chk_sse2_un) R-> (2024.0) 0x04c32df0 __memmove_chk
==18183== WARNING: new redirection conflicts with existing -- ignoring it
--18183-- old: 0x06800e80 (__memcpy_chk_sse2_un) R-> (2030.0) 0x04c33420 __memcpy_chk
--18183-- new: 0x06800e80 (__memcpy_chk_sse2_un) R-> (2024.0) 0x04c32df0 __memmove_chk
--18183-- REDIR: 0x67ff330 (libc.so.6:rindex) redirected to 0x4c2e7f0 (rindex)
--18183-- REDIR: 0x67fd650 (libc.so.6:strlen) redirected to 0x4c2eda0 (strlen)
--18183-- REDIR: 0x67f7f10 (libc.so.6:malloc) redirected to 0x4c2bb40 (malloc)
--18183-- REDIR: 0x6800e90 (libc.so.6:memcpy@GLIBC_2.2.5) redirected to 0x4c300c0 (memcpy@GLIBC_2.2.5)
--18183-- REDIR: 0x67fbc40 (libc.so.6:__GI_strcmp) redirected to 0x4c2fe70 (__GI_strcmp)
--18183-- REDIR: 0x67f8900 (libc.so.6:calloc) redirected to 0x4c2db10 (calloc)
--18183-- REDIR: 0x67fb9e0 (libc.so.6:__GI_strchr) redirected to 0x4c2e950 (__GI_strchr)
--18183-- REDIR: 0x6800e80 (libc.so.6:__memcpy_chk_sse2_unaligned) redirected to 0x4c33420 (__memcpy_chk)
--18183-- REDIR: 0x680c520 (libc.so.6:__strcmp_sse2_unaligned) redirected to 0x4c2fe20 (strcmp)
--18183-- REDIR: 0x67f8510 (libc.so.6:free) redirected to 0x4c2cd70 (free)
--18183-- REDIR: 0x68c0f80 (libc.so.6:__memcmp_sse4_1) redirected to 0x4c31ca0 (__memcmp_sse4_1)
--18183-- REDIR: 0x67f85c0 (libc.so.6:realloc) redirected to 0x4c2dd50 (realloc)
--18183-- REDIR: 0x67fdaa0 (libc.so.6:__GI_strncmp) redirected to 0x4c2f550 (__GI_strncmp)
--18183-- REDIR: 0x67fff50 (libc.so.6:__GI_strstr) redirected to 0x4c33590 (__strstr_sse2)
--18183-- REDIR: 0x68005f0 (libc.so.6:memchr) redirected to 0x4c30000 (memchr)
--18183-- REDIR: 0x680c4d0 (libc.so.6:__GI_strncpy) redirected to 0x4c2f150 (__GI_strncpy)
--18183-- REDIR: 0x68a2d20 (libc.so.6:__strpbrk_sse42) redirected to 0x4c33670 (strpbrk)
--18183-- REDIR: 0x6807710 (libc.so.6:strchrnul) redirected to 0x4c32e60 (strchrnul)
--18183-- REDIR: 0x6800e70 (libc.so.6:__GI_mempcpy) redirected to 0x4c33060 (__GI_mempcpy)
--18183-- REDIR: 0x68015a0 (libc.so.6:__GI_stpcpy) redirected to 0x4c31f80 (__GI_stpcpy)
--18183-- Reading syms from /lib/x86_64-linux-gnu/libnss_files-2.24.so
--18183-- Considering /usr/lib/debug/.build-id/8a/db8c8cdde0e4c3c3babb8874ae69dca15a666a.debug ..
--18183-- .. build-id is valid
--18183-- REDIR: 0x67fd0d0 (libc.so.6:__GI_strcpy) redirected to 0x4c2ef50 (__GI_strcpy)
--18183-- REDIR: 0x6801300 (libc.so.6:__GI_memset) redirected to 0x4c324c0 (memset)
--18183-- REDIR: 0x689f060 (libc.so.6:__strcasecmp_sse42) redirected to 0x4c2f6a0 (strcasecmp)
--18183-- REDIR: 0x6811170 (libc.so.6:__strcpy_sse2_unaligned) redirected to 0x4c2ee80 (strcpy)
--18183-- REDIR: 0x68a0c10 (libc.so.6:__strncasecmp_sse42) redirected to 0x4c2f780 (strncasecmp)
--18183-- REDIR: 0x6800980 (libc.so.6:__GI_memcmp) redirected to 0x4c31b40 (__GI_memcmp)
--18183-- REDIR: 0x6807500 (libc.so.6:rawmemchr) redirected to 0x4c32e90 (rawmemchr)
--18183-- REDIR: 0x68117a0 (libc.so.6:__strncpy_sse2_unaligned) redirected to 0x4c2f3b0 (__strncpy_sse2_unaligned)
--18183-- REDIR: 0x689e080 (libc.so.6:__strncmp_sse42) redirected to 0x4c2f630 (__strncmp_sse42)
--18183-- REDIR: 0x68a2be0 (libc.so.6:__strcspn_sse42) redirected to 0x4c336d0 (strcspn)failed to increase max number of open files from 1024 to 5000
--18183-- REDIR: 0x6815810 (libc.so.6:__strstr_sse2_unaligned) redirected to 0x4c33520 (strstr)
--18183-- REDIR: 0x67fd7f0 (libc.so.6:strnlen) redirected to 0x4c2ed40 (strnlen)
--18183-- Reading syms from /lib/x86_64-linux-gnu/libnss_dns-2.24.so
--18183-- Considering /usr/lib/debug/.build-id/b6/5aa56f4cecc441fce7f85f0b4aa0f2815e4e34.debug ..
--18183-- .. build-id is valid
--18183-- REDIR: 0x68a2e50 (libc.so.6:__strspn_sse42) redirected to 0x4c33730 (strspn)
hmm, nothing to indicate a fault in that list, rare.
run it outside of valgrind and get the core file, or run it in gdb directly. I would like to see the stack trace of this failure case.
karl.
mylselgan
commented
5 years ago Hi, Sorry I don't understand 'get the core file, or run it in gdb directly' could you please guide me by giving the command lines here
karlheyes
commented
5 years ago gdb --args icecast -c icecast/ice_auth.xml gdb> r
when fails
gdb> thread apply all bt
mylselgan
commented
5 years ago When I run under valgrind it runs without any issue for many days (i terminated to test under gdb)
here is the gdb report (I just executed "thread apply all bt" when icecast crashed and copied the output)
Thread 4 "icecast" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffef34f700 (LWP 16920)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) thread apply all bt
Thread 5 (Thread 0x7fffef24e700 (LWP 16921)):
#0 0x00007ffff61d48bd in poll () at ../sysdeps/unix/syscall-template.S:84
#1 0x0000555555562af0 in wait_for_serversock () at connection.c:920
#2 accept_client () at connection.c:1026
#3 connection_thread (arg=arg@entry=0x0) at connection.c:1428
#4 0x000055555558f725 in _start_routine (arg=0x555556656e20) at thread.c:727
#5 0x00007ffff649b4a4 in start_thread (arg=0x7fffef24e700) at pthread_create.c:456
#6 0x00007ffff61ddd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
Thread 4 (Thread 0x7fffef34f700 (LWP 16920)):
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff612942a in __GI_abort () at abort.c:89
#2 0x000055555556a30b in source_queue_advance (client=0x7fffd0003570) at source.c:824
#3 0x000055555556dd79 in send_listener (client=0x7fffd0003570, source=0x5555566554a0) at source.c:1371
#4 send_to_listener (client=0x7fffd0003570) at source.c:1227
#5 0x00005555555739b5 in worker (arg=arg@entry=0x5555558081c0) at client.c:735
#6 0x000055555558f725 in _start_routine (arg=0x555555814cf0) at thread.c:727
#7 0x00007ffff649b4a4 in start_thread (arg=0x7fffef34f700) at pthread_create.c:456
#8 0x00007ffff61ddd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
Thread 3 (Thread 0x7fffef450700 (LWP 16919)):
#0 0x00007ffff61d48bd in poll () at ../sysdeps/unix/syscall-template.S:84
#1 0x000055555556365a in util_timed_wait_for_fd (fd=<optimized out>, timeout=<optimized out>) at util.c:93
#2 0x0000555555573767 in worker_wait (worker=worker@entry=0x555555807ef0) at client.c:618
#3 0x0000555555573acf in worker (arg=arg@entry=0x555555807ef0) at client.c:769
#4 0x000055555558f725 in _start_routine (arg=0x555555807fb0) at thread.c:727
#5 0x00007ffff649b4a4 in start_thread (arg=0x7fffef450700) at pthread_create.c:456
#6 0x00007ffff61ddd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
Thread 2 (Thread 0x7ffff7fd3700 (LWP 16918)):
#0 0x00007ffff61d48bd in poll () at ../sysdeps/unix/syscall-template.S:84
#1 0x000055555556365a in util_timed_wait_for_fd (fd=<optimized out>, timeout=timeout@entry=5000) at util.c:93
#2 0x000055555557367c in log_commit_thread (arg=arg@entry=0x0) at client.c:907
#3 0x000055555558f725 in _start_routine (arg=0x555555814cf0) at thread.c:727
#4 0x00007ffff649b4a4 in start_thread (arg=0x7ffff7fd3700) at pthread_create.c:456
#5 0x00007ffff61ddd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
Thread 1 (Thread 0x7ffff7fd4740 (LWP 16914)):
#0 0x00007ffff64a47fd in nanosleep () at ../sysdeps/unix/syscall-template.S:84
#1 0x000055555558f2f6 in thread_sleep (len=len@entry=1000000) at thread.c:687
#2 0x0000555555567fe3 in _slave_thread () at slave.c:1219
#3 slave_initialize () at slave.c:241
#4 0x000055555555e9f4 in server_process () at main.c:236
#5 0x000055555555a78d in main (argc=3, argv=0x7fffffffe528) at main.c:435
(gdb)
ok, will have to check what conditions led to that. If you have a core file then you could query some things but if you just did it on-the fly and have quit gdb then ok. There's a safety check tripping on a particular condition so it's just a matter of finding out how it got to that point.
karl.
franzwarning
commented
4 years ago Getting the same issue. I think in my case it has to do with the stream_auth url authentication. If I remove only that option from my config there doesn't seem to be any problem. If I leave it in, I can repro almost every time -- but the steps are a bit weird so forgive me....
1.) Start a stream (via Butt) Ogg/vorbis 128k 2.) Have external server use liquidsoap to transcode that stream (fdkaac, 128k) 3.) Kill original stream (via killsource.xsl?mount=/mountname or via source) 4.) Error occurs.
<mount>
<mount-name>/*</mount-name>
<hidden>1</hidden>
<limit-rate>130k</limit-rate>
<authentication type="url">
<option name="auth_header" value="icecast-auth-user: 1"/>
<option name="mount_add" value="https://blah.com/icecast_mount_add/"/>
<option name="mount_remove" value="https://blah.com/icecast_mount_remove/"/>
<option name="stream_auth" value="https://blah.com/icecast_stream_auth/"/> <!-- Commenting this out makes everything work -->
<option name="listener_add" value="https://blah.com/listener_add/"/>
<option name="listener_remove" value="https://blah.com/listener_remove/"/>
<option name="allow_duplicate_users" value="1"/>
</authentication>
</mount>Dump when error occurs:
Thread 3 "icecast" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffebc7d700 (LWP 14143)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) thread apply all bt
Thread 5 (Thread 0x7fffeba7b700 (LWP 14145)):
#0 0x00007ffff67bcbf9 in __GI___poll (fds=fds@entry=0x7fffeba7aa40, nfds=3, timeout=timeout@entry=4000) at ../sysdeps/unix/sysv/linux/poll.c:29
#1 0x00005555555623f0 in poll (__timeout=4000, __nfds=<optimized out>, __fds=0x7fffeba7aa40) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
#2 wait_for_serversock () at connection.c:920
#3 accept_client () at connection.c:1026
#4 connection_thread (arg=arg@entry=0x0) at connection.c:1428
#5 0x00005555555910be in _start_routine (arg=0x555555804f60) at thread.c:727
#6 0x00007ffff6aa06db in start_thread (arg=0x7fffeba7b700) at pthread_create.c:463
#7 0x00007ffff67c988f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
Thread 4 (Thread 0x7fffebb7c700 (LWP 14144)):
#0 0x00007ffff67bcbf9 in __GI___poll (fds=fds@entry=0x7fffebb7bb70, nfds=nfds@entry=1, timeout=61) at ../sysdeps/unix/sysv/linux/poll.c:29
#1 0x0000555555562fae in poll (__timeout=<optimized out>, __nfds=1, __fds=0x7fffebb7bb70) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
#2 util_timed_wait_for_fd (fd=<optimized out>, timeout=<optimized out>) at util.c:93
#3 0x00005555555738e7 in worker_wait (worker=worker@entry=0x555555804ce0) at client.c:618
#4 0x0000555555573c56 in worker (arg=arg@entry=0x555555804ce0) at client.c:769
#5 0x00005555555910be in _start_routine (arg=0x555555804da0) at thread.c:727
#6 0x00007ffff6aa06db in start_thread (arg=0x7fffebb7c700) at pthread_create.c:463
#7 0x00007ffff67c988f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
Thread 3 (Thread 0x7fffebc7d700 (LWP 14143)):
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff66e8801 in __GI_abort () at abort.c:79
#2 0x0000555555573e45 in worker (arg=arg@entry=0x555555804a80) at client.c:704
#3 0x00005555555910be in _start_routine (arg=0x555555804b40) at thread.c:727
---Type <return> to continue, or q <return> to quit---
#4 0x00007ffff6aa06db in start_thread (arg=0x7fffebc7d700) at pthread_create.c:463
#5 0x00007ffff67c988f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
Thread 2 (Thread 0x7ffff7fcc700 (LWP 14142)):
#0 0x00007ffff67bcbf9 in __GI___poll (fds=fds@entry=0x7ffff7fcbbe0, nfds=nfds@entry=1, timeout=timeout@entry=5000) at ../sysdeps/unix/sysv/linux/poll.c:29
#1 0x0000555555562fae in poll (__timeout=5000, __nfds=1, __fds=0x7ffff7fcbbe0) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
#2 util_timed_wait_for_fd (fd=<optimized out>, timeout=timeout@entry=5000) at util.c:93
#3 0x00005555555737c3 in log_commit_thread (arg=arg@entry=0x0) at client.c:907
#4 0x00005555555910be in _start_routine (arg=0x5555558044c0) at thread.c:727
#5 0x00007ffff6aa06db in start_thread (arg=0x7ffff7fcc700) at pthread_create.c:463
#6 0x00007ffff67c988f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
Thread 1 (Thread 0x7ffff7fcfd40 (LWP 14138)):
#0 0x00007ffff6aaac60 in __GI___nanosleep (requested_time=requested_time@entry=0x7fffffffe330, remaining=remaining@entry=0x7fffffffe340) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
#1 0x0000555555590bf9 in thread_sleep (len=len@entry=1000000) at thread.c:687
#2 0x0000555555567cbb in _slave_thread () at slave.c:1219
#3 slave_initialize () at slave.c:241
#4 0x000055555555e6b4 in server_process () at main.c:236
#5 0x000055555555a13d in main (argc=3, argv=0x7fffffffe4f8) at main.c:435I don't see how the stream auth setting can affect the a sanity check that your core dump is reporting. If the issue is down to a previous fix then this won't appear with the master tree. I think the only crash fix since kh13 is relating to the on-connect type calls. I've just tried a simple run through here but have not triggered anything, but if it is still triggering then I may need ssh access to look at the core file as the issue is relating to a check with worker rebalancing. Best to open a separate report in such a case.
karl
OS: Debian Stretch (GNU/Linux 4.9.0-8-amd64 x86_64 ) Linux 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u4 (2018-08-21) x86_64 Icecast version: Icecast 2.4.0-kh12-20190416123413
I am running relay on 6 mounts (4 relays with url auth and 2 relays without auth) all 6 sources are aacp at 128kbps
this works well for 20 to 25 minuets then the crash happens
I received the following error with loglevel 1
Segmentation fault (core dumped) icecast -c icecast/ice.xmlSo i increased the log level to 4 then received the following error when crash happens