I am experiencing a deadlock situation where the backend (vnsiserver) freezes. No other clients are able to connect. Besides this i noticed, that the streamdev-server was still available. So i decided to take a deeper look at the problem.
After starting vdr using valgrinds tool helgrind, i found out a resource lock ordering violation where two mutexes wre locked in a different order.
`==29775==
==29775== Thread #9: lock order "0x2F4350 before 0x14CE02C8" violated
==29775==
==29775== Observed (incorrect) order is: acquisition of lock at 0x14CE02C8
==29775== at 0x48402CA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
==29775== by 0x269BC2: cMutex::Lock() (thread.c:259)
==29775== by 0x26A3A2: cMutexLock::Lock(cMutex) (thread.c:447)
==29775== by 0x57D2636: cVNSIClient::EpgChange() (vnsiclient.c:245)
==29775== by 0x57EE39B: cVNSIStatus::Action() (status.c:267)
==29775== by 0x26A214: cThread::StartThread(cThread) (thread.c:336)
==29775== by 0x4842B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
==29775== by 0x48A2608: start_thread (pthread_create.c:477)
==29775== by 0x4E40292: clone (clone.S:95)
==29775==
==29775== followed by a later acquisition of lock at 0x2F4350
==29775== at 0x484198F: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
==29775== by 0x26A022: cRwLock::Lock(bool, int) (thread.c:190)
==29775== by 0x26C952: cStateLock::Lock(cStateKey&, bool, int) (thread.c:775)
==29775== by 0x1B04FA: cChannels::GetChannelsRead(cStateKey&, int) (channels.c:851)
==29775== by 0x57D26A1: cChannels_Lock (channels.h:260)
==29775== by 0x57D26A1: cVNSIClient::EpgChange() (vnsiclient.c:251)
==29775== by 0x57EE39B: cVNSIStatus::Action() (status.c:267)
==29775== by 0x26A214: cThread::StartThread(cThread*) (thread.c:336)
==29775== by 0x4842B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
==29775== by 0x48A2608: start_thread (pthread_create.c:477)
==29775== by 0x4E40292: clone (clone.S:95)
==29775== Thread #1: lock order "0x14CE02C8 before 0x2F4350" violated
==29775==
==29775== Observed (incorrect) order is: acquisition of lock at 0x2F4350
==29775== at 0x484198F: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
==29775== by 0x26A022: cRwLock::Lock(bool, int) (thread.c:190)
==29775== by 0x26C952: cStateLock::Lock(cStateKey&, bool, int) (thread.c:775)
==29775== by 0x1B04FA: cChannels::GetChannelsRead(cStateKey&, int) (channels.c:851)
==29775== by 0x57D3C3C: cChannels_Lock (channels.h:260)
==29775== by 0x57D3C3C: cVNSIClient::processEPG_GetForChannel(cRequestPacket&) (vnsiclient.c:2490)
==29775== by 0x57D6051: cVNSIClient::processRequest(cRequestPacket&) (vnsiclient.c:579)
==29775== by 0x57D637C: cVNSIClient::Action() (vnsiclient.c:162)
==29775== by 0x26A214: cThread::StartThread(cThread) (thread.c:336)
==29775== by 0x4842B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
==29775== by 0x48A2608: start_thread (pthread_create.c:477)
==29775== by 0x4E40292: clone (clone.S:95)
==29775==
==29775== followed by a later acquisition of lock at 0x14CE02C8
==29775== at 0x48402CA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
==29775== by 0x269BC2: cMutex::Lock() (thread.c:259)
==29775== by 0x26A3A2: cMutexLock::Lock(cMutex) (thread.c:447)
==29775== by 0x57CBD83: non-virtual thunk to cVNSIClient::ChannelChange(cChannel const) (vnsiclient.c:391)
==29775== by 0x25C15F: cStatus::MsgChannelChange(cChannel const) (status.c:29)
==29775== by 0x1A9BF4: main (vdr.c:1071)`
I was using vdr-2.4.4 together with vnsiserver git a06ad5fbd4f5d2d547a17db291178d113dfbc699
Hi,
I am experiencing a deadlock situation where the backend (vnsiserver) freezes. No other clients are able to connect. Besides this i noticed, that the streamdev-server was still available. So i decided to take a deeper look at the problem.
After starting vdr using valgrinds tool helgrind, i found out a resource lock ordering violation where two mutexes wre locked in a different order. `==29775== ==29775== Thread #9: lock order "0x2F4350 before 0x14CE02C8" violated ==29775== ==29775== Observed (incorrect) order is: acquisition of lock at 0x14CE02C8 ==29775== at 0x48402CA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==29775== by 0x269BC2: cMutex::Lock() (thread.c:259) ==29775== by 0x26A3A2: cMutexLock::Lock(cMutex) (thread.c:447) ==29775== by 0x57D2636: cVNSIClient::EpgChange() (vnsiclient.c:245) ==29775== by 0x57EE39B: cVNSIStatus::Action() (status.c:267) ==29775== by 0x26A214: cThread::StartThread(cThread) (thread.c:336) ==29775== by 0x4842B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==29775== by 0x48A2608: start_thread (pthread_create.c:477) ==29775== by 0x4E40292: clone (clone.S:95) ==29775== ==29775== followed by a later acquisition of lock at 0x2F4350 ==29775== at 0x484198F: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==29775== by 0x26A022: cRwLock::Lock(bool, int) (thread.c:190) ==29775== by 0x26C952: cStateLock::Lock(cStateKey&, bool, int) (thread.c:775) ==29775== by 0x1B04FA: cChannels::GetChannelsRead(cStateKey&, int) (channels.c:851) ==29775== by 0x57D26A1: cChannels_Lock (channels.h:260) ==29775== by 0x57D26A1: cVNSIClient::EpgChange() (vnsiclient.c:251) ==29775== by 0x57EE39B: cVNSIStatus::Action() (status.c:267) ==29775== by 0x26A214: cThread::StartThread(cThread*) (thread.c:336) ==29775== by 0x4842B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==29775== by 0x48A2608: start_thread (pthread_create.c:477) ==29775== by 0x4E40292: clone (clone.S:95)
==29775== Thread #1: lock order "0x14CE02C8 before 0x2F4350" violated ==29775== ==29775== Observed (incorrect) order is: acquisition of lock at 0x2F4350 ==29775== at 0x484198F: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==29775== by 0x26A022: cRwLock::Lock(bool, int) (thread.c:190) ==29775== by 0x26C952: cStateLock::Lock(cStateKey&, bool, int) (thread.c:775) ==29775== by 0x1B04FA: cChannels::GetChannelsRead(cStateKey&, int) (channels.c:851) ==29775== by 0x57D3C3C: cChannels_Lock (channels.h:260) ==29775== by 0x57D3C3C: cVNSIClient::processEPG_GetForChannel(cRequestPacket&) (vnsiclient.c:2490) ==29775== by 0x57D6051: cVNSIClient::processRequest(cRequestPacket&) (vnsiclient.c:579) ==29775== by 0x57D637C: cVNSIClient::Action() (vnsiclient.c:162) ==29775== by 0x26A214: cThread::StartThread(cThread) (thread.c:336) ==29775== by 0x4842B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==29775== by 0x48A2608: start_thread (pthread_create.c:477) ==29775== by 0x4E40292: clone (clone.S:95) ==29775== ==29775== followed by a later acquisition of lock at 0x14CE02C8 ==29775== at 0x48402CA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==29775== by 0x269BC2: cMutex::Lock() (thread.c:259) ==29775== by 0x26A3A2: cMutexLock::Lock(cMutex) (thread.c:447) ==29775== by 0x57CBD83: non-virtual thunk to cVNSIClient::ChannelChange(cChannel const) (vnsiclient.c:391) ==29775== by 0x25C15F: cStatus::MsgChannelChange(cChannel const) (status.c:29) ==29775== by 0x1A9BF4: main (vdr.c:1071)`
I was using vdr-2.4.4 together with vnsiserver git a06ad5fbd4f5d2d547a17db291178d113dfbc699
Best regards, Max