SIPp / sipp

The SIPp testing tool
https://sipp.readthedocs.io
Other
916 stars 380 forks source link

segfault exitting SIPp when using rtpstream with multiple calls started at same time #699

Open peter-oneill opened 6 months ago

peter-oneill commented 6 months ago

Hit a segfault when SIPp exits after running calls using RTPstream. Backtrace from gdb:

(gdb) bt
#0  0x000000000043c624 in rtpstream_playback_thread (params=0x7ffff7f2f6e0) at /sipp/src/rtpstream.cpp:1328
#1  0x000000000059e878 in start (p=0x7ffff605fef0) at src/thread/pthread_create.c:207
#2  0x000000000059fa47 in __clone () at src/thread/x86_64/clone.s:22
Backtrace stopped: frame did not save the PC

invoked as gdb --args ./sipp-2e74b74b3b3bd076ce2c0d969916fc139ac008f9-debug -sf /app/dcluac_transcoding_rtp.xml -d 4000 -s 0123450000 <redacted IP1> -i <redacted IP2> -mi <redacted IP3> -r 10

Consistently reproduces with high enough call rate (e.g. 10 cps for me). Doesn't repro with 1cps. Intermittent at in between call rates.

Hittable at commit 2e74b74b3b3bd076ce2c0d969916fc139ac008f9, not at commit e55f87137c6ea58547cfbe060a8dacb5f833e368

peter-oneill commented 6 months ago

From a code skim, not sure how this is really intended to work: I think rtpstream_shutdown sets the exit_flag on every thread to 1, then immediately frees the memory. But rtpstream_playback_thread keeps references to threaddata for some time after exit_flag is last checked. Naively it looks like a race condition and is fundamentally unsafe.

But I don't seehow the timeout for a select call in rtp_playrtptask would affect this, so clearly I don't understand the area enough :-)