open-webrtc-toolkit / owt-server

General server (streaming/conference/transcoding/anayltics) for OWT. (A.k.a. MediaServer)
https://01.org/open-webrtc-toolkit
Apache License 2.0
1.13k stars 369 forks source link

videoMixer-sw.node randomly crash #1143

Open zhouchifelix opened 2 years ago

zhouchifelix commented 2 years ago

Hi, I encontered an issue that the videoMixer node would crash randomly when the videoMixer closed

Below is the steps: 1.lunch owt server 2.a owtnative client join the room, publish stream, and mix stream to common view

  1. leave the room 4.repeat step2 and step3 many times
  2. the videoagent will crash suddenly

The 1 is the crash log : Operating system: Linux 0.0.0 Linux 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 CPU: amd64 family 6 model 63 stepping 2 8 CPUs

GPU: UNKNOWN

Crash reason: SIGABRT Crash address: 0xd48 Process uptime: not available

Thread 0 (crashed) 0 libc.so.6 + 0x3ee97 rax = 0x0000000000000000 rdx = 0x0000000000000000 rcx = 0xffffffffffffffff rbx = 0x0000000000000000 rsi = 0x00007ffc51148460 rdi = 0x0000000000000002 rbp = 0x00007fa12f0797d8 rsp = 0x00007ffc51148460 r8 = 0x0000000000000000 r9 = 0x00007ffc51148460 r10 = 0x0000000000000008 r11 = 0x0000000000000246 r12 = 0x00007fa0f70d8640 r13 = 0x00007fa0f70d867d r14 = 0x0000000000000074 r15 = 0x0000000003ed7570 rip = 0x00007fa12ef00e97 Found by: given as instruction pointer in context 1 libc.so.6 + 0x8863f rsp = 0x00007ffc51148480 rip = 0x00007fa12ef4a63f Found by: stack scanning 2 libc.so.6 + 0x40801 rsp = 0x00007ffc51148580 rip = 0x00007fa12ef02801 Found by: stack scanning 3 libc.so.6 + 0x1bb1d0 rsp = 0x00007ffc51148650 rip = 0x00007fa12f07d1d0 Found by: stack scanning 4 libc.so.6 + 0x1b77d8 rsp = 0x00007ffc51148660 rip = 0x00007fa12f0797d8 Found by: stack scanning 5 libc.so.6 + 0x1b3d14 rsp = 0x00007ffc51148678 rip = 0x00007fa12f075d14 Found by: stack scanning 6 libc.so.6 + 0x1b77d8 rsp = 0x00007ffc51148688 rip = 0x00007fa12f0797d8 Found by: stack scanning 7 videoMixer-sw.node!_fini + 0xeb8 rsp = 0x00007ffc51148690 rip = 0x00007fa0f70d8640 Found by: stack scanning 8 videoMixer-sw.node!_fini + 0xef5 rsp = 0x00007ffc51148698 rip = 0x00007fa0f70d867d Found by: stack scanning 9 libc.so.6 + 0x3039a rsp = 0x00007ffc511486b0 rip = 0x00007fa12eef239a Found by: stack scanning 10 videoMixer-sw.node!_fini + 0xef5 rsp = 0x00007ffc511486d8 rip = 0x00007fa0f70d867d Found by: stack scanning 11 videoMixer-sw.node!_fini + 0xeb8 rsp = 0x00007ffc511486e0 rip = 0x00007fa0f70d8640 Found by: stack scanning 12 videoMixer-sw.node!_fini + 0x2c98 rsp = 0x00007ffc511486f0 rip = 0x00007fa0f70da420 Found by: stack scanning 13 libc.so.6 + 0x30412 rsp = 0x00007ffc51148700 rip = 0x00007fa12eef2412 Found by: stack scanning 14 videoMixer-sw.node!boost::condition_variable::~condition_variable() [condition_variable_fwd.hpp : 116 + 0x1f] rsp = 0x00007ffc51148730 rip = 0x00007fa0f6e9480f Found by: stack scanning 15 videoMixer-sw.node!boost::shared_mutex::~shared_mutex() [shared_mutex.hpp : 180 + 0x1e] rsp = 0x00007ffc51148750 rip = 0x00007fa0f6e94b51 Found by: stack scanning 16 videoMixer-sw.node!owt_base::VCMFrameEncoder::~VCMFrameEncoder() [VCMFrameEncoder.cpp : 109 + 0xc] rsp = 0x00007ffc51148770 rip = 0x00007fa0f6eb1eec Found by: stack scanning 17 videoMixer-sw.node!boost::unique_lock::lock() [lock_types.hpp : 346 + 0x5] rsp = 0x00007ffc51148780 rip = 0x00007fa0f6e97394 Found by: stack scanning 18 videoMixer-sw.node!owt_base::VCMFrameEncoder::~VCMFrameEncoder() [VCMFrameEncoder.cpp : 139 + 0x5] rsp = 0x00007ffc511487b0 rip = 0x00007fa0f6eb1fe1 Found by: stack scanning 19 videoMixer-sw.node!boost::detail::sp_counted_base::release() [sp_counted_base_std_atomic.hpp : 110 + 0x6] rsp = 0x00007ffc511487d0 rip = 0x00007fa0f6ea0242 Found by: stack scanning 20 videoMixer-sw.node!std::_Rb_tree<int, std::pair<int const, mcu::VideoFrameMixerImpl::Output>, std::_Select1st<std::pair<int const, mcu::VideoFrameMixerImpl::Output> >, std::less, std::allocator<std::pair<int const, mcu::VideoFrameMixerImpl::Output> > >::erase(int const&) [sp_counted_base_std_atomic.hpp : 108 + 0x10] rsp = 0x00007ffc511487f0 rip = 0x00007fa0f6ea557b Found by: stack scanning 21 videoMixer-sw.node!mcu::VideoFrameMixerImpl::removeOutput(int) [stl_map.h : 1062 + 0x5] rsp = 0x00007ffc51148840 rip = 0x00007fa0f6ea56dc Found by: stack scanning 22 videoMixer-sw.node!boost::detail::sp_counted_impl_pd<boost::asio::io_service::work*, boost::detail::sp_ms_deleter >::~sp_counted_impl_pd() [sp_counted_impl.hpp : 146 + 0xc] rsp = 0x00007ffc51148860 rip = 0x00007fa0f6e94700 Found by: stack scanning 23 0x7ffc51148850 rbx = 0x0000000000000000 rbp = 0x00007fa0f6e94700 rsp = 0x00007ffc51148868 rip = 0x00007ffc51148850 Found by: call frame info 24 videoMixer-sw.node!boost::shared_mutex::unlock() [lock_types.hpp : 331 + 0x9] rsp = 0x00007ffc51148880 rip = 0x00007fa0f6e975ff Found by: stack scanning 25 videoMixer-sw.node!mcu::VideoMixer::removeOutput(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) [VideoMixer.cpp : 216 + 0x9] rsp = 0x00007ffc511488b0 rip = 0x00007fa0f6ea1bce Found by: stack scanning 26 node + 0x691658 rsp = 0x00007ffc51148900 rip = 0x0000000000a91658 Found by: stack scanning 27 videoMixer-sw.node!VideoMixer::removeOutput(v8::FunctionCallbackInfo const&) [VideoMixerWrapper.cc : 307 + 0x8] rsp = 0x00007ffc511489a0 rip = 0x00007fa0f6e7a293 Found by: stack scanning 28 node + 0x553760 rsp = 0x00007ffc51148a10 rip = 0x0000000000953760 Found by: stack scanning 29 videoMixer-sw.node!VideoMixer::clearText(v8::FunctionCallbackInfo const&) [new_allocator.h : 125 + 0xf] rsp = 0x00007ffc51148a40 rip = 0x00007fa0f6e7a170 Found by: stack scanning 30 node + 0x696503 rsp = 0x00007ffc51148a60 rip = 0x0000000000a96503 Found by: stack scanning 31 node + 0x553760 rsp = 0x00007ffc51148aa0 rip = 0x0000000000953760 Found by: stack scanning 32 videoMixer-sw.node!VideoMixer::clearText(v8::FunctionCallbackInfo const&) [new_allocator.h : 125 + 0xf] rsp = 0x00007ffc51148ac0 rip = 0x00007fa0f6e7a170 Found by: stack scanning 33 node + 0xae6507 rsp = 0x00007ffc51148ad0 rip = 0x0000000000ee6507 Found by: stack scanning 34 videoMixer-sw.node!VideoMixer::clearText(v8::FunctionCallbackInfo const&) [new_allocator.h : 125 + 0xf] rsp = 0x00007ffc51148b50 rip = 0x00007fa0f6e7a170 Found by: stack scanning 35 node + 0x70d72c rsp = 0x00007ffc51148b60 rip = 0x0000000000b0d72c Found by: stack scanning 36 node + 0x1364470 rsp = 0x00007ffc51148b98 rip = 0x0000000001764470 Found by: stack scanning 37 node + 0x70e37f rsp = 0x00007ffc51148c40 rip = 0x0000000000b0e37f Found by: stack scanning 38 node + 0xc37c59 rsp = 0x00007ffc51148c60 rip = 0x0000000001037c59 Found by: stack scanning 39 node + 0xc37c20 rsp = 0x00007ffc51148c68 rip = 0x0000000001037c20 Found by: stack scanning 40 node + 0x70e2d0 rsp = 0x00007ffc51148c80 rip = 0x0000000000b0e2d0 Found by: stack scanning

Thread 1 0 libpthread.so.0 + 0xd9f3 rax = 0xfffffffffffffe00 rdx = 0x0000000000000000 rcx = 0xffffffffffffffff rbx = 0x0000000003578b58 rsi = 0x0000000000000080 rdi = 0x0000000003578b84 rbp = 0x0000000003578b7c rsp = 0x00007fa12edbeda0 r8 = 0x0000000000000001 r9 = 0x0000000000000000 r10 = 0x0000000000000000 r11 = 0x0000000000000246 r12 = 0x0000000003578b84 r13 = 0x0000000000000000 r14 = 0x0000000003578b30 r15 = 0x000000000000020b rip = 0x00007fa12f2c09f3 Found by: given as instruction pointer in context 1 libpthread.so.0 + 0xd690 rsp = 0x00007fa12edbede8 rip = 0x00007fa12f2c0690 Found by: stack scanning 2 node + 0x5c5c89 rsp = 0x00007fa12edbee70 rip = 0x00000000009c5c89 Found by: stack scanning 3 node + 0x522634 rsp = 0x00007fa12edbee80 rip = 0x0000000000922634 Found by: stack scanning 4 libpthread.so.0 + 0x76db rsp = 0x00007fa12edbeec0 rip = 0x00007fa12f2ba6db Found by: stack scanning 5 libc.so.6 + 0x12188f rsp = 0x00007fa12edbef80 rip = 0x00007fa12efe388f Found by: stack scanning

starwarfan commented 2 years ago

Hi, Which version you find this issue?

zhouchifelix commented 2 years ago

Hi, Which version you find this issue?

Hi, In branch 4.2.x

starwarfan commented 2 years ago

I didn't find this in our test. Your log shows that there was a "removeOutput" call after "clearText". Have you done some customization in video mixer?