Tri-State Cored after 0ver 40 days uptime

[tsawyer]

Tri-State runs only the DAHDI/pseudo channel driver. Of course there have been no updates to this server in the uptime. But it will be interesting to see what caused the crash. Maybe it's something that has already been fixed.

core-asterisk-2023-06-08T23-12-59Z-full.txt

[tsawyer]

Here is /var/log/asterisk/messages.log at the time of the crash

[2023-06-08 16:12:58.114] ERROR[60972] channel.c: FRACK!, Failed assertion 0 (0)
[2023-06-08 16:12:58.354] ERROR[60972] : Got 8 backtrace records
# 0: [0x563292ec098d] asterisk utils.c:2798 __ast_assert_failed()
# 1: [0x563292d51e19] asterisk utils.h:732 _ast_assert()
# 2: [0x563292d5b6e3] asterisk channel.c:3065 ast_waitfor_nandfds()
# 3: [0x563292d5bdfc] asterisk channel.c:3160 ast_waitfor_n()
# 4: [0x7f017821b3e7] app_rpt.so app_rpt.c:3996 rpt()
# 5: [0x563292ebd89c] asterisk utils.c:1574 dummy_start()
# 6: [0x7f017bc4aea7] libpthread.so.0 pthread_create.c:478 start_thread()
# 7: [0x7f017b865a2f] libc.so.6 accept.c:26 __libc_accept()

[tsawyer]

Any initial thought as to what caused this?

[InterLinked1]

This looks like the same underlying issue as #159, which I had not seen before prior to that.

[tsawyer]

Tri-state crored again. This time after running for almost 30 days. Is there something we should be doing to find this sooner than waiting nearly a month?

core-asterisk-2023-09-30T06-13-50Z-full.txt

Here's a snip of the log around the time of the core at line 65:

messages.log

[InterLinked1]

This seems to be a separate issue that I haven't seen before:

Thread 1 (Thread 0x7f9ada53f700 (LWP 288193)):
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
        set = {__val = {134238211, 0 <repeats 13 times>, 3140312, 140303064623219}}
        pid = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
#1  0x00007f9afbd99537 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0}, sa_mask = {__val = {0 <repeats 13 times>, 94468519138800, 140303627708803, 140303064623888}}, sa_flags = -68774251, sa_restorer = 0xe246c}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x000055eb28ea18fd in ast_do_crash () at utils.c:2770
No locals.
#3  0x000055eb28cf1846 in my_do_crash () at astmm.c:230
No locals.
#4  0x000055eb28cf2271 in __ast_free (ptr=0x7f9ae4127280, file=0x7f9adb928f5d "iax2/parser.c", lineno=1333, func=0x7f9adb929278 <__PRETTY_FUNCTION__.0> "iax_frame_free") at astmm.c:488
        reg = 0x0
#5  0x00007f9adb91b418 in iax_frame_free (fr=0x7f9ae4127280) at iax2/parser.c:1333
        __PRETTY_FUNCTION__ = "iax_frame_free"
#6  0x00007f9adb8c9235 in iax2_frame_free (fr=0x7f9ae4127280) at chan_iax2.c:2141
        __FUNCTION__ = "iax2_frame_free"
#7  0x00007f9adb8cf273 in __attempt_transmit (data=0x7f9ae4127280) at chan_iax2.c:3633
        f = 0x7f9ae4127280
        freeme = 1
        callno = 14786
        __PRETTY_FUNCTION__ = "__attempt_transmit"
        __FUNCTION__ = "__attempt_transmit"
#8  0x00007f9adb9009ac in iax2_process_thread (data=0x7f9ad40fb060) at chan_iax2.c:12187