abumq / easyloggingpp

C++ logging library. It is powerful, supports asynchronous low latency, extendable, light-weight, fast performing, thread and type safe and consists of many built-in features. It provides ability to write logs in your own customized format. It also provide support for logging your classes, third-party libraries, STL and third-party containers etc.
MIT License
3.79k stars 927 forks source link

Segmentation Fault with multiple threads v9.96.7 like issue #580 #686

Open ace88sg opened 5 years ago

ace88sg commented 5 years ago

Hi, I'm getting segmentation fault when running multiple loggers via multiple threads. This is similar to issue #580 I have used the new version v.9.96.7 which supposedly fixes this issue since v 9.96.4 (stated in #580 ) I have also implemented the

define ELPP_NO_GLOBAL_LOCK

but the segmentation fault still persist randomly

My Configuration implements

define ELPP_THREAD_SAFE

 #define ELPP_UNICODE
 #define ELPP_UTC_DATETIME
 #define ELPP_FORCE_USE_STD_THREAD
 #define ELPP_NO_DEFAULT_LOG_FILE
 #define ELPP_NO_GLOBAL_LOCK
 INITIALIZE_EASYLOGGINGPP

I've also tried implementing the ELPP_THREAD_SAFE option via the compiler as in issue #314 but it doesn't fix the problem

On Failure gdb show this ... Running v.9.96.7

Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffa7fff700 (LWP 2379)] el::base::DefaultLogDispatchCallback::dispatch(std::string&&) (this=0x0, this@entry=0xaffe40, logLine=logLine@entry=<unknown type in /root/Documents/MagicEye/SUSI/testLib2/testLib2, CU 0x24caec, DIE 0x2f7c28>) at easylogging++.cc:2240 2240 if (m_data->logMessage()->logger()->m_typedConfigurations->toStandardOutput(m_data->logMessage()->level())) {

 ---------------------more info about the failure-------------------------------------

BackTrace

0 el::base::DefaultLogDispatchCallback::dispatch(std::string&&) (this=0x0, this@entry=0xaffe40,

logLine=logLine@entry=<unknown type in /root/Documents/MagicEye/SUSI/testLib2/testLib2, CU 0x24caec, DIE 0x2f7c28>)
at easylogging++.cc:2240

1 0x0000000000529946 in el::base::DefaultLogDispatchCallback::handle (this=0xaffe40,

this@entry=<error reading variable: Cannot access memory at address 0x7fffa7ffe718>, data=<optimized out>) at easylogging++.cc:2213

================================================= It always fail at the same location.

Previously with version 9.95.3, it will fail at the same location as in #580 Program terminated with signal SIGSEGV, Segmentation fault.

0 el::base::DefaultLogDispatchCallback::dispatch(std::string&&) (this=0xfb40f0,

logLine=...) at ../uart_server/lib/easyloggingpp/src/easylogging++.cc:2132

Any Ideas about sorting the issue out? Thanks in advance ...

Regards Rod

amafarinha commented 5 years ago

I'm facing the same issue, the output of gdb's backtrace is:

0 0x000000000052245c in el::LogMessage::level() const ()

1 0x000000000051b9cd in el::base::DefaultLogDispatchCallback::dispatch(std::__cxx11::basic_string<char, std::char_traits, std::allocator >&&) ()

2 0x000000000051b799 in el::base::DefaultLogDispatchCallback::handle(el::LogDispatchData const*) ()

It also happens to me that it prints some DEBUG and TRACE logs although they are disable in the configs... as in #700

justinasvd commented 4 years ago

I am running into the same issue in v9.96.7 as well. This happens sporadically, but when it happens it's invariably looks like this:


=================================================================
==4280==ERROR: AddressSanitizer: SEGV on unknown address 0x00000000000c (pc 0x5580d6251cfa bp 0x7ffe031aaa90 sp 0x7ffe031aaa50 T0)
==4280==The signal is caused by a READ memory access.
==4280==Hint: address points to the zero page.
    #0 0x5580d6251cf9 in std::unordered_map<el::Level, unsigned int, std::hash<el::Level>, std::equal_to<el::Level>, std::allocator<std::pair<el::Level const, unsigned int> > >::find(el::Level const&) /usr/include/c++/7/bits/unordered_map.h:923
    #1 0x5580d6251cf9 in el::Logger::isFlushNeeded(el::Level) include/vse/logging/easylogging++.h:2260
    #2 0x5580d6251cf9 in el::base::DefaultLogDispatchCallback::dispatch(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&) src/logging/easylogging++.cc:2230
    #3 0x5580d6252f99 in el::base::DefaultLogDispatchCallback::handle(el::LogDispatchData const*) src/logging/easylogging++.cc:2212
    #4 0x5580d62519ef in el::base::LogDispatcher::dispatch() src/logging/easylogging++.cc:2499
    #5 0x5580d62560f8 in el::base::Writer::triggerDispatch() src/logging/easylogging++.cc:2632
    #6 0x5580d6255cc4 in el::base::Writer::processDispatch() src/logging/easylogging++.cc:2613
    #7 0x5580d60599dd in el::base::Writer::~Writer() ../libVSE/include/vse/logging/easylogging++.h:3203```
fengcanyue commented 4 years ago

I'm also using v9.96.7 ,This occurs during long runs.My project is Qt project,. My configure:

#easylogging 
DEFINES += ELPP_QT_LOGGING  \
           ELPP_STL_LOGGING \
           ELPP_NO_DEFAULT_LOG_FILE \
           ELPP_FEATURE_CRASH_LOG \
           ELPP_NO_GLOBAL_LOCK \
           ELPP_THREAD_SAFE

When segmentation fault happens, the backtrace is always look like this:

2020-06-09 01:17:53,892 [DEBUG] void UdpWorker::recvMsg():43 Recv: 2a2a486561642a2a524f43504f572a2a0400000001000000710500000800000030303a34353a35362a2a524f43504f572a2a5461696c2a2a
2020-06-09 01:17:53,893 [DEBUG] void UdpWorker::recvMsg():47 Recv:  MainType = 4    nSubType = 1    nMsgValue = 1393    nMsgDataLen = 8 pMsgData = 00:45:56
2020-06-09 01:17:54,258 [DEBUG] void UdpWorker::recvMsg():43 Recv: 2a2a486561642a2a524f43504f572a2a050000000000000000000000000000002a2a524f43504f572a2a5461696c2a2a
2020-06-09 01:17:54,259 [DEBUG] void UdpWorker::recvMsg():47 Recv:  MainType = 5    nSubType = 0    nMsgValue = 0   nMsgDataLen = 0 pMsgData = 
2020-06-09 01:17:54,896 [DEBUG] void UdpWorker::recvMsg():43 Recv: 2a2a486561642a2a524f43504f572a2a0400000001000000710500000800000030303a34353a35372a2a524f43504f572a2a5461696c2a2a
2020-06-09 01:17:54,896 [DEBUG] void UdpWorker::recvMsg():47 Recv:  MainType = 4    nSubType = 1    nMsgValue = 1393    nMsgDataLen = 8 pMsgData = 00:45:57
2020-06-09 01:17:55,929 [INFO] void logRotator():242 About to rotate log file!
2020-06-09 01:17:55,943 [DEBUG] void UdpWorker::recvMsg():43 Recv: 2a2a486561642a2a524f43504f572a2a0400000001000000720500000800000030303a34353a35382a2a524f43504f572a2a5461696c2a2a
2020-06-09 01:17:56,219 [FATAL] void el::base::debug::logCrashReason(int, bool, el::Level, const char*):2876 Recv: 2a2a486561642a2a524f43504f572a2a0400000001000000720500000800000030303a34353a35382a2a524f43504f572a2a5461696c2a2aCRASH HANDLED; Application has crashed due to [SIGSEGV] signal
    ======= Backtrace: =========
    [1] /nand/ROCPOW/app/Director() [0xb2c7c] 
    [2] /nand/ROCPOW/app/Director() [0xb2df4] 
    [3]  0xb6bebed0]:_vfpv3_neon/libc.so.6(__default_sa_restorer+0) [0xb6bebed0]
    [4] /nand/ROCPOW/app/Director() [0xb78fc] 
    [5] /nand/ROCPOW/app/Director() [0xb0048] 
    [6] /nand/ROCPOW/app/Director() [0xafe30] 
    [7] /nand/ROCPOW/app/Director() [0xb1068] 
    [8] /nand/ROCPOW/app/Director() [0xb1c80] 
    [9] /nand/ROCPOW/app/Director() [0xb1a70] 
    [10] /nand/ROCPOW/app/Director() [0x63928] 
    [11] /nand/ROCPOW/app/Director() [0x66714] 
    [12] /nand/ROCPOW/app/Director() [0x108db0] 
    [13] /nand/ROCPOW/app/Director() [0x997d4c] 
    [14] /nand/ROCPOW/app/Director() [0x834f84] 
    [15] /nand/ROCPOW/app/Director() [0x840ea4] 
    [16] /nand/ROCPOW/app/Director() [0x1b2f5c] 
    [17] /nand/ROCPOW/app/Director() [0x1b9eac] 
    [18] /nand/ROCPOW/app/Director() [0x982884] 
    [19] /nand/ROCPOW/app/Director() [0x9af274] 
    [20] /nand/ROCPOW/app/Director() [0x9af4b4] 
    [21] /nand/ROCPOW/app/Director() [0x9af8ac] 
    [22] /nand/ROCPOW/app/Director() [0x9813f0] 
    [23] /nand/ROCPOW/app/Director() [0x9816e8] 
    [24] /nand/ROCPOW/app/Director() [0x88ae60] 
    [25] /nand/ROCPOW/app/Director() [0x88d7cc] 

2020-06-09 01:17:56,221 [WARNING] void el::base::debug::logCrashReason(int, bool, el::Level, const char*):2876 Aborting application. Reason: Fatal log at [3rdparty/easyloggingpp/easylogging++.cc:2876]

I use addr2line tool to translates ,it look like this:

lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ ls
Director  Director.bak  Director.bak.0
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x88d7cc
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x88ae60
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x9816e8
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x9813f0
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x9af8ac
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x9af4b4
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x9af274
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x982884
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x1b9eac
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x1b2f5c
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x840ea4
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x834f84
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x997d4c
??:?
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x108db0
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/moc/moc_UdpWorker.cpp:55
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x66714
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/util/UdpWorker.cpp:44 (discriminator 8)
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0x63928
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/./3rdparty/easyloggingpp/easylogging++.h:3202
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0xb1a70
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/3rdparty/easyloggingpp/easylogging++.cc:2613
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0xb1c80
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/3rdparty/easyloggingpp/easylogging++.cc:2632 (discriminator 2)
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0xb1068
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/3rdparty/easyloggingpp/easylogging++.cc:2501
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0xafe30
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/3rdparty/easyloggingpp/easylogging++.cc:2213 (discriminator 4)
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0xb0048
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/3rdparty/easyloggingpp/easylogging++.cc:2230 (discriminator 2)
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0xb78fc
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/3rdparty/easyloggingpp/easylogging++.h:2260
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ addr2line -e Director.bak 0xb2df4
/home/lu/share/Lubo-linuxDir/B400/branches/bug-fix/3rdparty/easyloggingpp/easylogging++.cc:2888
lu@lu-PC:~/share/Lubo-linuxDir/B400/branches/bug-fix/bin$ 

What can I do to solve this problem?