search

ZhuChaozheng / libjingle

Automatically exported from code.google.com/p/libjingle
0 stars 0 forks source link

Deadlock in talk_base MessageQueue/Thread #389

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. talk_base::Thread A holds lock to A's crit_ and tries to get lock of 
MessageQueueHandler::crit_, thread B holds MessageQueueHandler::crit_ and tries 
to acquire A's crit_

What version of the product are you using? On what operating system?

URL: http://libjingle.googlecode.com/svn/trunk/talk/base
Repository Root: http://libjingle.googlecode.com/svn
Repository UUID: dd674b97-3498-5ee5-1854-bdd07cd0ff33
Revision: 115
Node Kind: directory
Schedule: normal
Last Changed Author: ronghuawu@google.com
Last Changed Rev: 115
Last Changed Date: 2012-02-08 22:30:21 +0100 (Wed, 08 Feb 2012)
MacBook-Pro.local 12.1.0 Darwin Kernel Version 12.1.0: Tue Aug 14 13:29:55 PDT 
2012; root:xnu-2050.9.2~1/RELEASE_X86_64 x86_64

Please provide any additional information below.

Thread B stack:

#0  0x00007fff8e2c8122 in __psynch_mutexwait ()
#1  0x00007fff8aefbddd in pthread_mutex_lock ()
#2  0x00000001003e371a in talk_base::CriticalSection::Enter (this=0x1026515e8) 
at criticalsection.h:103
#3  0x00000001003e36c1 in talk_base::CritScope::CritScope (this=0x10aaf29f0, 
pcrit=0x1026515e8) at criticalsection.h:134
#4  0x00000001003e14eb in talk_base::CritScope::CritScope (this=0x10aaf29f0, 
pcrit=0x1026515e8) at criticalsection.h:135
#5  0x00000001003e5f73 in talk_base::MessageQueueManager::Clear 
(this=0x1026515d0, handler=0x10391e500) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/messagequeue.c
c:99
#6  0x00000001003e5a83 in talk_base::MessageHandler::~MessageHandler 
(this=0x10391e500) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/messagehandler
.cc:34
#7  0x0000000100424acf in talk_base::StreamInterface::~StreamInterface 
(this=0x10391e500) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/stream.cc:54
#8  0x0000000100420cdf in talk_base::SocketStream::~SocketStream 
(this=0x10391e500) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/socketstream.c
c:38
#9  0x0000000100420c03 in talk_base::SocketStream::~SocketStream 
(this=0x10391e500) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/socketstream.c
c:36
#10 0x0000000100420baa in talk_base::SocketStream::~SocketStream 
(this=0x10391e500) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/socketstream.c
c:36
#11 0x00000001003d08c3 in 
talk_base::DisposeData<talk_base::StreamInterface>::~DisposeData 
(this=0x103e27600) at messagequeue.h:127
#12 0x00000001003d07e3 in 
talk_base::DisposeData<talk_base::StreamInterface>::~DisposeData 
(this=0x103e27600) at messagequeue.h:127
#13 0x00000001003d083a in 
talk_base::DisposeData<talk_base::StreamInterface>::~DisposeData 
(this=0x103e27600) at messagequeue.h:127
#14 0x00000001003e6ab6 in talk_base::MessageQueue::Get (this=0x103967360, 
pmsg=0x10aaf2df8, cmsWait=1000, process_io=true) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/messagequeue.c
c:220
#15 0x00000001004357ca in talk_base::Thread::ProcessMessages (this=0x103967360, 
cmsLoop=1000) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/thread.cc:480
#16 0x00000001000ebe48 in cortc::CortcapiMideeAccessServer::DoWork 
(this=0x103967300) at /Users/sz/COVO/cortc/cortcapi/cortcapi_ma.cpp:13
#17 0x000000010040a810 in talk_base::SignalThread::Run (this=0x103967300) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/signalthread.c
c:152
#18 0x000000010040d1ba in talk_base::SignalThread::Worker::Run 
(this=0x103967360) at signalthread.h:124
#19 0x00000001004355a8 in talk_base::Thread::PreRun (pv=0x10395d240) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/thread.cc:331
#20 0x00007fff8aef6782 in _pthread_start ()
#21 0x00007fff8aee31c1 in thread_start ()
(gdb) thr 13

Thread A stack:

#0  0x00007fff8e2c8122 in __psynch_mutexwait ()
#1  0x00007fff8aefbddd in pthread_mutex_lock ()
#2  0x00000001003e371a in talk_base::CriticalSection::Enter (this=0x1039673f8) 
at criticalsection.h:103
#3  0x00000001003e36c1 in talk_base::CritScope::CritScope (this=0x10efea8d0, 
pcrit=0x1039673f8) at criticalsection.h:134
#4  0x00000001003e14eb in talk_base::CritScope::CritScope (this=0x10efea8d0, 
pcrit=0x1039673f8) at criticalsection.h:135
#5  0x0000000100435dc9 in talk_base::Thread::Clear (this=0x103967360, 
phandler=0x10380aac0, id=4294967295, removed=0x0) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/thread.cc:441
#6  0x00000001003e5feb in talk_base::MessageQueueManager::Clear 
(this=0x1026515d0, handler=0x10380aac0) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/messagequeue.c
c:102
#7  0x00000001003e5a83 in talk_base::MessageHandler::~MessageHandler 
(this=0x10380aac0) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/messagehandler
.cc:34
#8  0x0000000100424acf in talk_base::StreamInterface::~StreamInterface 
(this=0x10380aac0) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/stream.cc:54
#9  0x0000000100420cdf in talk_base::SocketStream::~SocketStream 
(this=0x10380aac0) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/socketstream.c
c:38
#10 0x0000000100420c03 in talk_base::SocketStream::~SocketStream 
(this=0x10380aac0) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/socketstream.c
c:36
#11 0x0000000100420baa in talk_base::SocketStream::~SocketStream 
(this=0x10380aac0) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/socketstream.c
c:36
#12 0x000000010041b0c9 in talk_base::ReuseSocketPool::~ReuseSocketPool 
(this=0x104942cd8) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/socketpool.cc:
171
#13 0x000000010041afd3 in talk_base::ReuseSocketPool::~ReuseSocketPool 
(this=0x104942cd8) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/socketpool.cc:
169
#14 0x0000000100394055 in talk_base::AsyncHttpRequest::~AsyncHttpRequest 
(this=0x104942a00) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/asynchttpreque
st.cc:58
#15 0x0000000100393f83 in talk_base::AsyncHttpRequest::~AsyncHttpRequest 
(this=0x104942a00) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/asynchttpreque
st.cc:57
#16 0x0000000100393f2a in talk_base::AsyncHttpRequest::~AsyncHttpRequest 
(this=0x104942a00) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/asynchttpreque
st.cc:57
#17 0x000000010040ce08 in talk_base::SignalThread::EnterExit::~EnterExit 
(this=0x10efeabc0) at signalthread.h:145
#18 0x000000010040aba3 in talk_base::SignalThread::EnterExit::~EnterExit 
(this=0x10efeabc0) at signalthread.h:141
#19 0x000000010040a772 in talk_base::SignalThread::OnMessage (this=0x104942a00, 
msg=0x10efeae58) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/signalthread.c
c:149
#20 0x0000000100394781 in talk_base::AsyncHttpRequest::OnMessage 
(this=0x104942a00, message=0x10efeae58) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/asynchttpreque
st.cc:102
#21 0x00000001003947df in non-virtual thunk to 
talk_base::AsyncHttpRequest::OnMessage(talk_base::Message*) () at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/asynchttpreque
st.cc:105
#22 0x00000001003e752b in talk_base::MessageQueue::Dispatch (this=0x1017a5450, 
pmsg=0x10efeae58) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/messagequeue.c
c:372
#23 0x0000000100435804 in talk_base::Thread::ProcessMessages (this=0x1017a5450, 
cmsLoop=-1) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/thread.cc:482
#24 0x0000000100435718 in talk_base::Thread::Run (this=0x1017a5450) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/thread.cc:342
#25 0x00000001004355a8 in talk_base::Thread::PreRun (pv=0x1017830d0) at 
/Users/sz/COVO/cortc/trunk/third_party/libjingle/source/talk/base/thread.cc:331
#26 0x00007fff8aef6782 in _pthread_start ()
#27 0x00007fff8aee31c1 in thread_start ()

Original issue reported on code.google.com by ser...@comoyo.com on 1 Sep 2012 at 12:01

GoogleCodeExporter commented 9 years ago 
While thinking about general solution using this workaround in 
talk_base::MessageQueue::Get()
...
        if (MQID_DISPOSE == pmsg->message_id) {
          ASSERT(NULL == pmsg->phandler);
          crit_.Leave(); // HACK: locked crit_ will cause a deadlock when called MessageQueueHandler calls Thread::Clear()
          delete pmsg->pdata;
          crit_.Enter(); // TODO: fix deadlock
          continue;
        }

Original comment by ser...@comoyo.com on 1 Sep 2012 at 12:21

GoogleCodeExporter commented 9 years ago 
better solution is to stash to be deleted objects in a vector and do actual 
delete when crit_ is released

Original comment by ser...@comoyo.com on 14 Feb 2013 at 10:12

GoogleCodeExporter commented 9 years ago 
Fixed in r311:

 2   seanegan         if (MQID_DISPOSE == pmsg->message_id) {
   311 mallinath@           // Delete the object, but *not inside the crit scope!*.
   311 mallinath@           deleter.messages.push_back(*pmsg);
   311 mallinath@           // To be safe, make sure we don't return this message.
   311 mallinath@           *pmsg = Message();
     2   seanegan           continue;
     2   seanegan         }
     2   seanegan         return true;

Original comment by ser...@comoyo.com on 22 Jul 2013 at 12:12