Improve robustness to stack overflow

I performed couple of tests to asses the stack overflow crash behavior
the user nicknamed pogo11 described. It is very important to capture
stack overflow correctly in my application as I needed to lower the
amount of stack allocated per thread. The reason being that the stack
size allocated for a single WIN32 thread by default is defined in the
EXE header and I cannot modify the 3rd party code which creates plenty
of threads internally. If I leave the thread stack size defined in the
EXE header high, then the total of the reserved stacks for all threads
eats up a considerable amount from the 2GB of the virtual process
space on 32bit architecture. We are writing geometric algorithms,
where recursion is natural. It is quite difficult to estimate whether
the reserved stack size is sufficient or not. It would be very
unfortunate, if the application closed without any warning or crash
information.

So I modified your CrashRptTest with following recursive function to
test the crash overflows. I am using Visual Studio 2010.

struct DisableTailOptimization
{
       ~DisableTailOptimization() {
               ++ v;
       }
       static int v;
};

int DisableTailOptimization::v = 0;

static void CauseStackOverflow()
{
       DisableTailOptimization v;
       CauseStackOverflow();
}

The excercise with the DisableTailOptimization struct does exactly
what the name stands for. A simple recursive function would be
optimized out completely replaced with a loop. Having a destructor of
a local class instance being called after the recursive function call
effectively disables tail optimization in release mode. Without the
DisableTailOptimization instance the function call ends up in endless
loop in release mode. In debug mode it always causes stack overflow.

When testing the crash reporting, the application just closes on stack
overflow structured exception when compiled in debug mode. The stack
overflow is correctly reported by CrashRpt when running in release
mode. My explanation being, the optimized CrashRpt library requires
much less of the stack space than the CrashRpt library compiled in
debug mode, therefore the tiny bit of the stack left when the CPU
refuses to decrease the ESP register is sufficient to execute the
error reporting. The debug version requires more stack than available,
therefore the "poof".

I verified the solution offered by pogo11 to execute the error
reporting on a newly instantiated thread and it works even in debug
mode. I suppose it makes it a better solution from the robustness
point of view. The solution of pogo11 makes the crash reporting of the
stack overflow work independent of the compiler optimization settings.
I am using following piece of code:

//Vojtech: Based on martin.bis...@gmail.com comment in
// 
http://groups.google.com/group/crashrpt/browse_thread/thread/a1dbcc56acb58b27/fb
d0151dd8e26daf?lnk=gst&q=stack+overflow#fbd0151dd8e26daf
// Thread procedure doing the dump for stack overflow.
DWORD WINAPI StackOverflowThreadFunction(LPVOID threadParameter)
{
   PEXCEPTION_POINTERS pExceptionPtrs =
reinterpret_cast<PEXCEPTION_POINTERS>(threadParameter);
       CCrashHandler *pCrashHandler =
CCrashHandler::GetCurrentProcessCrashHandler();
       ATLASSERT(pCrashHandler != NULL);
       if (pCrashHandler != NULL) {
               // Acquire lock to avoid other threads (if exist) to crash while we
are inside.
               pCrashHandler->CrashLock(TRUE);
               CR_EXCEPTION_INFO ei;
               memset(&ei, 0, sizeof(CR_EXCEPTION_INFO));
               ei.cb = sizeof(CR_EXCEPTION_INFO);
               ei.exctype = CR_SEH_EXCEPTION;
               ei.pexcptrs = pExceptionPtrs;
               pCrashHandler->GenerateErrorReport(&ei);
               // Terminate process
               TerminateProcess(GetCurrentProcess(), 1);
       }
   return 0;
}

// Structured exception handler
LONG WINAPI CCrashHandler::SehHandler(PEXCEPTION_POINTERS
pExceptionPtrs)
{
       //Vojtech: Based on martin.bis...@gmail.com comment in
       //
http://groups.google.com/group/crashrpt/browse_thread/thread/a1dbcc56acb58b27/fb
d0151dd8e26daf?lnk=gst&q=stack+overflow#fbd0151dd8e26daf
       if (pExceptionPtrs != 0 && pExceptionPtrs->ExceptionRecord != 0 &&
pExceptionPtrs->ExceptionRecord->ExceptionCode ==
EXCEPTION_STACK_OVERFLOW) {
       // Special case to handle the stack overflow exception.
       // The dump will be realized from another thread.
       // Create another thread that will do the dump.
       HANDLE thread = ::CreateThread(0, 0,
&StackOverflowThreadFunction, pExceptionPtrs, 0, 0);
       ::WaitForSingleObject(thread, INFINITE);
       ::CloseHandle(thread);
       // Terminate process
       TerminateProcess(GetCurrentProcess(), 1);
   }

..... here goes the original code.
Original issue reported on code.google.com by zexspect...@gmail.com on 11 Dec 2011 at 1:01
jirentabu / crashrpt

Improve robustness to stack overflow #121
