hazelcast / hazelcast-cpp-client

Hazelcast IMDG C++ Client
https://hazelcast.com/clients/cplusplus/
Apache License 2.0
79 stars 44 forks source link

Hazelcast Hang during shutdown #372

Open KalebNeguisse opened 6 years ago

KalebNeguisse commented 6 years ago

Hi

Hazelcast seems to hang when we do a hazelcast shutdown either explicitly or by deleting HazelcastClient

HazelcastClient hazelcastClient = (HazelcastClient)m_hazelcastClient; delete hazelcastClient;

I see the following stack trace (It seems to be stuck at wakeup)

ntdll!NtReleaseKeyedEvent+0xa ntdll!longjmp+0xf080 HazelcastClient3_8_3_64!hazelcast::util::Thread::wakeup+0x30 HazelcastClient3_8_3_64!hazelcast::client::spi::PartitionService::shutdown+0x17 HazelcastClient3_8_3_64!hazelcast::client::spi::LifecycleService::shutdown+0x6e HazelcastClient3_8_3_64!hazelcast::client::HazelcastClient::~HazelcastClient+0x2d VdrHazelcastWrapper!VdrHazelcastWrapper::invalidateConnection+0x69 VdrHazelcastWrapper!VdrHazelcastWrapper::~VdrHazelcastWrapper+0x39 regini!ProfileIsFTSFeatureAvailable+0x245e ucrtbase!time64+0x167 ucrtbase!register_onexit_function+0x87 ucrtbase!execute_onexit_table+0x31 regini!ProfileIsFTSFeatureAvailable+0x10c22 regini!ProfileIsFTSFeatureAvailable+0x10d18 ntdll!LdrShutdownProcess+0x1db ntdll!RtlExitUserProcess+0x90 ucrtbase!wassert+0x356 ucrtbase!wassert+0x2ae rdsarcrt+0xc0ee9 kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d

Can somebody help?

Thank you and loikking forward to your answer.

Kaleb

KalebNeguisse commented 6 years ago

Here is another stack trace showing a slightly different type of hang

ntdll!NtReleaseKeyedEvent+0xa ntdll! ?? ::FNODOBFM::`string'+0x4db0 HazelcastClient3_8_3_64!hazelcast::client::connection::CallPromise::setResponse+0x1f5 HazelcastClient3_8_3_64!hazelcast::client::connection::CallPromise::setException+0x2f HazelcastClient3_8_3_64!hazelcast::client::spi::InvocationService::cleanResources+0x439 HazelcastClient3_8_3_64!hazelcast::client::connection::Connection::close+0x25c HazelcastClient3_8_3_64!hazelcast::util::IOUtil::closeResource+0x35 HazelcastClient3_8_3_64!hazelcast::client::connection::ConnectionManager::shutdown+0x1df HazelcastClient3_8_3_64!hazelcast::client::spi::LifecycleService::shutdown+0x5d HazelcastClient3_8_3_64!hazelcast::client::HazelcastClient::~HazelcastClient+0x2d VdrHazelcastWrapper!VdrHazelcastWrapper::invalidateConnection+0x69 VdrHazelcastWrapper!VdrHazelcastWrapper::~VdrHazelcastWrapper+0x39 regini!VdrHazelcastStore::~VdrHazelcastStore+0x1fe ucrtbase!::operator()+0xb7 ucrtbase!crt_seh_guarded_call::operator()<, & __ptr64, >+0x3b ucrtbase!execute_onexit_table+0x31 regini!dllmain_crt_process_detach+0x4e regini!dllmain_dispatch+0xc0 ntdll!LdrShutdownProcess+0x1db ntdll!RtlExitUserProcess+0x90 ucrtbase!exit_or_terminate_process+0x46 ucrtbase!common_exit+0xf2 acreate!scrt_common_main_seh+0x12f kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d

ihsandemir commented 6 years ago

@KalebNeguisse What about other threads? Do you also have logs?

Are you on windows? (I see ntdll)

KalebNeguisse commented 6 years ago

Yes I am, on Windows.

ihsandemir commented 6 years ago

@KalebNeguisse Can you please share the stack traces of all threads if possible?

KalebNeguisse commented 6 years ago

@ihsandemir That is the only thread that there is at the time this happens. There are no other threads.