apache / logging-log4cxx

Apache Log4cxx is a C++ port of Apache Log4j
http://logging.apache.org/log4cxx
Apache License 2.0
273 stars 122 forks source link

Delay/pause/crash in LogManager::shutdown() #373

Closed hordi closed 5 months ago

hordi commented 6 months ago

Can't provide any simple app to reproduce the issue, but getting it frequently enough in production multi-thread code during LogManager::shutdown() call before main.cpp exit (nothing is running at this moment). Using 1.2.0 version. Very often see some strange delay ~1-30 seconds, sometimes up to 30 minutes. Sometimes crashed with ACCESS_VIOLATION message. In Process Explorer I saw some function with 0x0000000 address (disappeared quickly). Core dump was absolutely not helpful. Used logging is trivial enough - console low intensive output only.

rm5248 commented 6 months ago

It sounds like your other threads are not being shutdown before the main thread is exiting. That does lead to undefined behavior.

The only safe way to exit is to stop all threads before returning from main, since once the application is exiting the statically allocated variables will be destroyed in an undefined order. If other threads are still running, crashes are likely as other threads will try to access memory that has just be deallocated.

hordi commented 6 months ago

Yes, I have one more running thread which I can't turn off, that is intel-TBB thread that I can't control (no API provided to completely shutdown TBB, but no active work there during log3cxx shutdown).

swebb2066 commented 6 months ago

Could you build a Log4cxx with LOG4CXX_EVENTS_AT_EXIT=ON?

This will remove Log4cxx static variable destructors from the onexit chain. You could also then add logging to any other onexit processing to which you have access which may provide you some useful information

hordi commented 6 months ago

Finally have found how to turn off TBB thread - do not getting the crash in run log4cxx shutdown any more.

swebb2066 commented 6 months ago

@hordi Thank you for the update