Asterisk crashes when res_speech_unimrcp.so is loaded automatically on startup with asterisk in console with high verbosity, using sofia-sip 1.2.10devel from freeswitch trunk

[GoogleCodeExporter]

Hi there,

Here are some tests I have run to show that the Asterisk segmentation 
faults when the sofia-sip library from the freeswitch project is used 
(1.2.10devel). I present the results here for anyone who wishes to use 
asterisk/sofia-sip/unimrcp with this version of sofia-sip and show how to 
replicate the problem. See Test 3. Test 1 and 2 were successful.

I've used on a CentOS 5.4 (2.6.18-164.6.1.el5PAE, gcc 4.1.2) server:
http://unimrcp.googlecode.com/files/uni-ast-package.tar.gz

1) Test 1: Standard uni-ast-package.tar.gz:
From uni-ast-package.tar.gz I have installed:
 - Asterisk 1.6.10
 - APR 1.3.8
 - APR-Util 1.3.9
 - Sofia SIP 1.2.10
 - UniMRCP 0.8.0

The process I followed, was:
# ./ast-install.sh
# cd asterisk
# make samples
# cd ..
# ./uni-install.sh
# cd asterisk-unimrcp
# ./configure
# make
# make install

# asterisk -c
    core stop now
# asterisk -vvvvvvvvvvvvvvvc
    core stop now

No problems with this setup. I then uninstalled everything (APR, APR-Util, 
Sofia-SIP, UniMRCP, Asterisk).

2) Test 2: Standard uni-ast-package.tar.gz, but asterisk-1.6.0.18:
From 
http://downloads.asterisk.org/pub/telephony/asterisk/releases/asterisk-
1.6.0.18.tar.gz
 - Asterisk 1.6.0.18
From uni-ast-package.tar.gz I have installed:
 - APR 1.3.8
 - APR-Util 1.3.9
 - Sofia SIP 1.2.10
 - UniMRCP 0.8.0

The process I followed, was:
# mv asterisk asterisk.1.6.10
# wget 
http://downloads.asterisk.org/pub/telephony/asterisk/releases/asterisk-
1.6.0.18.tar.gz
# tar xzvf asterisk-1.6.0.18.tar.gz
# mv asterisk-1.6.0.18 asterisk
# ./ast-install.sh
# cd asterisk
# make samples
# cd ..
# ./uni-install.sh
# cd asterisk-unimrcp
# ./configure
# make
# make install

# asterisk -c
    core stop now
# asterisk -vvvvvvvvvvvvvvvc
    core stop now

No problems with this setup. I then uninstalled everything (APR, APR-Util, 
Sofia-SIP, UniMRCP, Asterisk).

3) Test 3: Standard uni-ast-package.tar.gz, but sofia-sip from the 
freeswitch trunk:
From uni-ast-package.tar.gz I have installed:
 - Asterisk 1.6.10
 - APR 1.3.8
 - APR-Util 1.3.9
 - UniMRCP 0.8.0
From http://svn.freeswitch.org/svn/freeswitch/trunk/libs/sofia-sip
 - Sofia SIP 1.2.10devel

The process I followed, was:
# ./ast-install.sh
# cd asterisk
# make samples
# cd ..
# cd unimrcp/libs
# mv sofia-sip sofia-sip.1.2.10
# svn co http://svn.freeswitch.org/svn/freeswitch/trunk/libs/sofia-sip
# chmod a+x autogen.sh
# Change libsofia-sip-ua/Makefile.am
    noinst_LTLIBRARIES = libsofia-sip-ua.la
        to
    lib_LTLIBRARIES = libsofia-sip-ua.la

    Patch file:

Index: libsofia-sip-ua/Makefile.am
===================================================================
--- libsofia-sip-ua/Makefile.am (revision 15723)
+++ libsofia-sip-ua/Makefile.am (working copy)
@@ -35,7 +35,7 @@

 DOXYGEN = doxygen

-noinst_LTLIBRARIES = libsofia-sip-ua.la
+lib_LTLIBRARIES = libsofia-sip-ua.la

 libsofia_sip_ua_la_SOURCES =
 libsofia_sip_ua_la_LIBADD =    bnf/libbnf.la \

# ./autogen.sh
# cd ../../
# ./uni-install.sh
# cd asterisk-unimrcp
# ./configure
# make
# make install

# asterisk -c
    core stop now
No problems with that test using low verbosity

# asterisk -vvvvvvvvvvvvvvvvvvvvvvvvvc
Segmentation fault (core dumped)

Segmentation fault occurs (place seems to be quite random, but lower down 
in call stack it is always in the load_modules function - loader.c:1050, 
SSL_accept as last entry on call stack). Below are backtraces for two 
attempts at running asterisk in the console. Running asterisk with 
valgrind, doesn't cause a segmentation fault (but it does obviously load 
very slowly, so race conditions won't occur that easily):

# valgrind --leak-check=full --show-reachable=yes --log-file=/root/uni-ast-
package.valgrind.log /usr/sbin/asterisk -vvvvvvvvvvvvvvvvvvvvvvvvvc

Also running asterisk by starting it up as a daemon, works fine as well:

# asterisk
# asterisk -r

Finally, adding "noload => res_speech_unimrcp.so" 
to /etc/asterisk/modules.conf, starting asterisk with (asterisk -
vvvvvvvvvvvvvvvvvvvvvvvvvc), and then after asterisk loaded successfully, 
typing "module load res_speech_unimrcp.so" works fine and causes no 
crashes.

So the segmentation fault occurs when verbosity is high and asterisk is 
started on the console with res_speech_unimrcp.so loaded automatically on 
start-up. So it looks like the problem somehow comes in with sofia-sip 
from the freeswitch trunk, perhaps some dependent library not yet properly 
initialized, I don't know. I'm not sure what is different between the 
sofia-sip version you are using and the one used in the freeswitch trunk.

Kind regards,
Derik Thirion

# asterisk -vvvvvvvvvvvvvvvvvvvvvvvvvc
Segmentation fault (core dumped)

#0  0x005305e9 in strcasecmp () from /lib/libc.so.6
#1  0x080e164f in ast_manager_register_struct (act=0xb40074c0) at 
manager.c:3385
#2  0x080e17c5 in ast_manager_register2 (action=0x2d1077f "SIPpeers", 
auth=513, func=0x2ca3350 <manager_sip_show_peers>, 
synopsis=0x2d11d9a "List SIP peers (text format)",
    description=0x2d22f60 "Description: Lists SIP peers in text format 
with details on current status.\nPeerlist will follow as separate events, 
followed by a final event called\nPeerlistComplete.\nVariables: \n  
ActionID: <id>\tAct"...) at manager.c:3424
#3  0x02cd5fd6 in load_module () at chan_sip.c:24564
#4  0x080da761 in start_resource (mod=0x8c0c2b8) at loader.c:726
#5  0x080db3e6 in load_resource_list (load_order=0xbfc55df0, 
global_symbols=0, mod_count=0xbfc55df8) at loader.c:904
#6  0x080db6c6 in load_modules (preload_only=0) at loader.c:1050
#7  0x080784c9 in main (argc=2, argv=0xbfc57314) at asterisk.c:3487
#8  0x004d4e9c in __libc_start_main () from /lib/libc.so.6
#9  0x0805c871 in SSL_accept ()
(gdb) frame 0
#0  0x005305e9 in strcasecmp () from /lib/libc.so.6
(gdb) frame 1
#1  0x080e164f in ast_manager_register_struct (act=0xb40074c0) at 
manager.c:3385
3385                    int ret = strcasecmp(cur->action, act->action);
(gdb) frame 2
#2  0x080e17c5 in ast_manager_register2 (action=0x2d1077f "SIPpeers", 
auth=513, func=0x2ca3350 <manager_sip_show_peers>, 
synopsis=0x2d11d9a "List SIP peers (text format)",
    description=0x2d22f60 "Description: Lists SIP peers in text format 
with details on current status.\nPeerlist will follow as separate events, 
followed by a final event called\nPeerlistComplete.\nVariables: \n  
ActionID: <id>\tAct"...) at manager.c:3424
3424            if (ast_manager_register_struct(cur)) {
(gdb) frame 3
#3  0x02cd5fd6 in load_module () at chan_sip.c:24564
24564           ast_manager_register2("SIPpeers", EVENT_FLAG_SYSTEM | 
EVENT_FLAG_REPORTING, manager_sip_show_peers,
(gdb) frame 4
#4  0x080da761 in start_resource (mod=0x8c0c2b8) at loader.c:726
726             res = mod->info->load();
(gdb) frame 5
#5  0x080db3e6 in load_resource_list (load_order=0xbfc55df0, 
global_symbols=0, mod_count=0xbfc55df8) at loader.c:904
904                     switch (start_resource(mod)) {
(gdb) frame 6
#6  0x080db6c6 in load_modules (preload_only=0) at loader.c:1050
1050            if ((res = load_resource_list(&load_order, 0, 
&modulecount)) < 0) {
(gdb) frame 7
#7  0x080784c9 in main (argc=2, argv=0xbfc57314) at asterisk.c:3487
3487            if (load_modules(0)) {
(gdb) frame 8
#8  0x004d4e9c in __libc_start_main () from /lib/libc.so.6
(gdb) frame 9
#9  0x0805c871 in SSL_accept ()

# asterisk -vvvvvvvvvvvvvvvvvvvvvvvvvc
Segmentation fault (core dumped)

#0  0x005305f3 in strcasecmp () from /lib/libc.so.6
#1  0x080f2261 in ast_register_application2 (app=0x15da0e "StackPop", 
execute=0x15cc60 <pop_exec>, synopsis=0x15d658 "Remove one address from 
gosub stack",
    description=0x15d618 "  StackPop():\nRemoves last label on the stack, 
discarding it.\n", mod=0xb7c135a0) at pbx.c:4568
#2  0x0015b46d in load_module () at app_stack.c:542
#3  0x080da761 in start_resource (mod=0xb7c135a0) at loader.c:726
#4  0x080db3e6 in load_resource_list (load_order=0xbfea1d90, 
global_symbols=0, mod_count=0xbfea1d98) at loader.c:904
#5  0x080db6c6 in load_modules (preload_only=0) at loader.c:1050
#6  0x080784c9 in main (argc=2, argv=0xbfea32b4) at asterisk.c:3487
#7  0x004d4e9c in __libc_start_main () from /lib/libc.so.6
#8  0x0805c871 in SSL_accept ()
(gdb) frame 0
#0  0x005305f3 in strcasecmp () from /lib/libc.so.6
(gdb) frame 1
#1  0x080f2261 in ast_register_application2 (app=0x15da0e "StackPop", 
execute=0x15cc60 <pop_exec>, synopsis=0x15d658 "Remove one address from 
gosub stack",
    description=0x15d618 "  StackPop():\nRemoves last label on the stack, 
discarding it.\n", mod=0xb7c135a0) at pbx.c:4568
4568                    if (!(res = strcasecmp(app, tmp->name))) {
(gdb) frame 2
#2  0x0015b46d in load_module () at app_stack.c:542
542             ast_register_application(app_pop, pop_exec, pop_synopsis, 
pop_descrip);
(gdb) frame 3
#3  0x080da761 in start_resource (mod=0xb7c135a0) at loader.c:726
726             res = mod->info->load();
(gdb) frame 4
#4  0x080db3e6 in load_resource_list (load_order=0xbfea1d90, 
global_symbols=0, mod_count=0xbfea1d98) at loader.c:904
904                     switch (start_resource(mod)) {
(gdb) frame 5
#5  0x080db6c6 in load_modules (preload_only=0) at loader.c:1050
1050            if ((res = load_resource_list(&load_order, 0, 
&modulecount)) < 0) {
(gdb) frame 6
#6  0x080784c9 in main (argc=2, argv=0xbfea32b4) at asterisk.c:3487
3487            if (load_modules(0)) {
(gdb) frame 7
#7  0x004d4e9c in __libc_start_main () from /lib/libc.so.6
(gdb) frame 8
#8  0x0805c871 in SSL_accept ()

Original issue reported on code.google.com by thirion...@gmail.com on 30 Nov 2009 at 9:27

[GoogleCodeExporter]

Hi there,

I've now run unimrcp trunk, unimrcp-deps-0.8.0's APR, APR-Util, asterisk-unimrcp
trunk and sofia-sip from 
http://unimrcp.googlecode.com/files/uni-ast-package.tar.gz.
Everything works 100% and I've not had a single segmentation fault since then.

Kind regards,
Derik

Original comment by thirion...@gmail.com on 2 Dec 2009 at 8:10

[GoogleCodeExporter]

Hi there,

Just out of curiosity, how did you create the sofia-sip in
http://unimrcp.googlecode.com/files/uni-ast-package.tar.gz? Is that directly 
from the
trunk of sofia-sip on sourceforge?

Kind regards,
Derik

Original comment by thirion...@gmail.com on 2 Dec 2009 at 8:14

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Hi Derik,

So I assume this issue indirectly relates to Issue-53. Probably we should wait 
a bit
more to come to final conclusion.
BTW, I still stick with the sofia-sip included in generic unimrcp-deps package.
Though I consider to upgrade in the future.

Original comment by achalo...@gmail.com on 2 Dec 2009 at 8:58

[GoogleCodeExporter]

Hi Arsen,

I agree. Let's wait a bit.

PS: the crash not only occurred with the res_speech_unimrcp module, but also my 
own
code. After using the "older" version of sofia-sip, everything was 100%, so I 
assume
the FreeSwitch guys did something which I don't fully understand there yet. 
Once I
have some time, I'll do a diff between the versions to try and understand what 
might
be causing this. Perhaps it is related to the initstate issues I saw earlier...

For now, I suppose, we can close the issue?

Kind regards,
Derik

Original comment by thirion...@gmail.com on 2 Dec 2009 at 9:54

[GoogleCodeExporter]

So I assume, this issue has been fixed with issue-53

Original comment by achalo...@gmail.com on 9 Dec 2009 at 4:56

• Changed state: Fixed
• Added labels: Component-Asterisk, OpSys-Linux, Priority-Medium, Type-Defect

[GoogleCodeExporter]

With the dependency package of unimrcp 0.9.0, it is fine. I'll continue testing 
it.

Original comment by thirion...@gmail.com on 6 Feb 2010 at 9:34

• Changed state: Verified