Compiler segfaults on Windows when exceeding a certain stacksize

[dsuedholt]

When using the faust compiler on windows system, compiling large processes like the ones created by the fds library results in a segmentation fault.

To reproduce, compile StiffString.dsp on windows.

[sletz]

This seems to be a stack size issue. On other platforms (Linux, macOS) we use POSIX API to setup a separated thread with a bigger stack size (see https://github.com/grame-cncm/faust/blob/master-dev/compiler/libcode.cpp#L184). We'll probably have to do the same on Windows using CreateThread.

[sletz]

@dsuedholt can you possibly try the following code (blind written code) ?:

static void callFun(compile_fun fun)
{
#if defined(EMCC)
    // No thread support in JS
    fun(NULL);
#elif defined(_WIN32)
    DWORD id;
    faustassert(CreateThread(NULL, MAX_STACK_SIZE, fun, NULL, 0, &id) != NULL);
#else
    pthread_t      thread;
    pthread_attr_t attr;
    faustassert(pthread_attr_init(&attr) == 0);
    faustassert(pthread_attr_setstacksize(&attr, MAX_STACK_SIZE) == 0);
    faustassert(pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE) == 0);
    faustassert(pthread_create(&thread, &attr, fun, nullptr) == 0);
    pthread_join(thread, nullptr);
#endif
}

[dsuedholt]

Doens't compile for me:

C:/Users/[...]/faust/compiler/libcode.cpp: In function 'void callFun(compile_fun)':
C:/Users/[...]/faust/compiler/libcode.cpp:190:52: error: invalid conversion from 'compile_fun' {aka 'void* (*)(void*)'} to 'LPTHREAD_START_ROUTINE' {aka 'long unsigned int (*)(void*)'} [-fpermissive]
  190 |     faustassert(CreateThread(NULL, MAX_STACK_SIZE, fun, NULL, 0, &id) != NULL);
      |                                                    ^~~
      |                                                    |
      |                                                    compile_fun {aka void* (*)(void*)}

[dfober]

it doesn't compile, the expected compile_fun should be DWORD WINAPI ThreadProc( In LPVOID lpParameter ); but it compiles with a cast to LPTHREAD_START_ROUTINE Next it doesn't seems to change anything. @dsuedholt https://github.com/dsuedholt could you also test?

Le dim. 14 mars 2021 à 10:46, Stéphane Letz @.***> a écrit :

@dsuedholt https://github.com/dsuedholt ca, you possibly try the following code (blind written code) ?:

static void callFun(compile_fun fun) {

if defined(EMCC) defined(_WIN32)

// No thread support in JS or WIN32
fun(NULL);

elif defined(_WIN32)

DWORD id;
faustassert(CreateThread(NULL, MAX_STACK_SIZE, fun, NULL, 0, &id) != NULL);

else

pthread_t      thread;
pthread_attr_t attr;
faustassert(pthread_attr_init(&attr) == 0);
faustassert(pthread_attr_setstacksize(&attr, MAX_STACK_SIZE) == 0);
faustassert(pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE) == 0);
faustassert(pthread_create(&thread, &attr, fun, nullptr) == 0);
pthread_join(thread, nullptr);

endif

}

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/grame-cncm/faust/issues/564#issuecomment-798877981, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJHQ4IJOKI6MYXOXXSSV3TDSAWJANCNFSM4ZEEHSPA .

[dfober]

Oups, I could have been more explicit: replace

faustassert(CreateThread(NULL, MAX_STACK_SIZE, fun, NULL, 0, &id) != NULL);

with

faustassert(CreateThread(NULL, MAX_STACK_SIZE, LPTHREAD_START_ROUTINE(fun), NULL, 0, &id) != NULL);

Le dim. 14 mars 2021 à 11:00, David Südholt @.***> a écrit :

Doens't compile for me:

C:/Users/[...]/faust/compiler/libcode.cpp: In function 'void callFun(compile_fun)': C:/Users/[...]/faust/compiler/libcode.cpp:190:52: error: invalid conversion from 'compile_fun' {aka 'void ()(void)'} to 'LPTHREAD_START_ROUTINE' {aka 'long unsigned int ()(void*)'} [-fpermissive] 190	faustassert(CreateThread(NULL, MAX_STACK_SIZE, fun, NULL, 0, &id) != NULL);	^~~
compile_fun {aka void ()(void*)}

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/grame-cncm/faust/issues/564#issuecomment-798879584, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJHQ7RXSEG6TNMULEPYWTTDSCLDANCNFSM4ZEEHSPA .

[dsuedholt]

@dfober This does compile but still gives a segfault, but in a different place now. Here's the gdb output with this change:

(gdb) r StiffString.dsp
Starting program: C:\msys64\usr\local\bin\faust.exe StiffString.dsp
[New Thread 25076.0x5934]
[New Thread 25076.0x58a8]
[New Thread 25076.0x1cfc]
[New Thread 25076.0x1418]

Thread 5 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 25076.0x1418]
0x00007ff749face88 in getProperty(CTree*, CTree*, CTree*&) ()
(gdb) thread apply all bt

Thread 5 (Thread 25076.0x1418):
#0  0x00007ff749face88 in getProperty(CTree*, CTree*, CTree*&) ()
#1  0x00007ff749eb6b75 in addLayerDef(CTree*, CTree*, CTree*) ()
#2  0x00007ff749eb713c in pushMultiClosureDefs(CTree*, CTree*, CTree*) ()
#3  0x00007ff749ebf431 in evalprocess(CTree*) ()
#4  0x00007ff749f575bf in evaluateBlockDiagram(CTree*, int&, int&) ()
#5  0x00007ff749f57cd8 in threadEvaluateBlockDiagram(void*) ()
#6  0x00007ffacefb7c24 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#7  0x00007ffad0b2d721 in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#8  0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 4 (Thread 25076.0x1cfc):
#0  0x00007ffad0b60724 in ntdll!ZwWaitForWorkViaWorkerFactory () from C:\Windows\SYSTEM32\ntdll.dll
#1  0x00007ffad0af3fe0 in ntdll!RtlInitializeResource () from C:\Windows\SYSTEM32\ntdll.dll
#2  0x00007ffacefb7c24 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#3  0x00007ffad0b2d721 in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#4  0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 3 (Thread 25076.0x58a8):
#0  0x00007ffad0b60724 in ntdll!ZwWaitForWorkViaWorkerFactory () from C:\Windows\SYSTEM32\ntdll.dll
#1  0x00007ffad0af3fe0 in ntdll!RtlInitializeResource () from C:\Windows\SYSTEM32\ntdll.dll
#2  0x00007ffacefb7c24 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#3  0x00007ffad0b2d721 in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#4  0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 2 (Thread 25076.0x5934):
#0  0x00007ffad0b60724 in ntdll!ZwWaitForWorkViaWorkerFactory () from C:\Windows\SYSTEM32\ntdll.dll
#1  0x00007ffad0af3fe0 in ntdll!RtlInitializeResource () from C:\Windows\SYSTEM32\ntdll.dll
#2  0x00007ffacefb7c24 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#3  0x00007ffad0b2d721 in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#4  0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 1 (Thread 25076.0x43c8):
#0  0x00007ffad0b0069b in ntdll!RtlFreeHeap () from C:\Windows\SYSTEM32\ntdll.dll
#1  0x00007ffad0bc5201 in ntdll!RtlpNtMakeTemporaryKey () from C:\Windows\SYSTEM32\ntdll.dll
#2  0x00007ffad0b01b6d in ntdll!RtlGetCurrentServiceSessionId () from C:\Windows\SYSTEM32\ntdll.dll
#3  0x00007ffad0b012e0 in ntdll!RtlGetCurrentServiceSessionId () from C:\Windows\SYSTEM32\ntdll.dll
#4  0x00007ffad0b006e1 in ntdll!RtlFreeHeap () from C:\Windows\SYSTEM32\ntdll.dll
#5  0x00007ffad0739cfc in msvcrt!free () from C:\Windows\System32\msvcrt.dll
#6  0x00007ff749f4cd6a in global::~global() ()
#7  0x00007ff749f4d55e in global::destroy() ()
#8  0x00007ff749f629b5 in compileFaustFactory(int, char const**, char const*, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, bool) ()
#9  0x00007ff74a1732b5 in main ()

And here is the gdb output from before the change;

This goes all the way down to over #5000, called from eval(CTree*, CTree*, CTree*) in evalprocess

[sletz]

Can you try to raise MAX_STACK_SIZE defined here https://github.com/grame-cncm/faust/blob/master-dev/compiler/evaluate/loopDetector.hh#L52 ?

[dsuedholt]

Tried with 2*, and 10*, still segfault

[dfober]

It looks like a Tree element is corrupted. As far back as I can find i.e. libcode.cpp:1154, in static Tree evaluateBlockDiagram(Tree expandedDefList, int& numInputs, int& numOutputs) expandedDefList fields are incorrect: at least fNext is an incorrect pointer (0xdddddddddddddddd) Maybe an issue with the thread context?

Le dim. 14 mars 2021 à 21:13, David Südholt @.***> a écrit :

Tried with 2, and 10, still segfault

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/grame-cncm/faust/issues/564#issuecomment-798971788, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJHQ2WDGLUVQER2SW5AK3TDUKEZANCNFSM4ZEEHSPA .

[sletz]

@dfober I tried that to possibly detect memory or address error:

diff --git a/build/CMakeLists.txt b/build/CMakeLists.txt
index be0b32399..cc1849dfb 100644
--- a/build/CMakeLists.txt
+++ b/build/CMakeLists.txt
@@ -151,7 +151,7 @@ if (MSVC)
        set (FAUST_DEFINITIONS ${FAUST_DEFINITIONS} -DMSVisualStudio)
 else()
        set (FAUST_DEFINITIONS ${FAUST_DEFINITIONS} -DLIBDIR="${LIBSDIR}")
-       set(CMAKE_CXX_FLAGS_DEBUG   "-g  -Wall -Wextra -Wno-unused-parameter -Wno-unused-function -Wno-overloaded-virtual -Wshadow")
+       set(CMAKE_CXX_FLAGS_DEBUG   "-g  -Wall -Wextra -Wno-unused-parameter -fsanitize=memory -fsanitize=address -Wno-unused-function -Wno-overloaded-virtual -Wshadow")
        set(CMAKE_CXX_FLAGS_RELEASE "-O3 -Wall -Wextra -Wno-unused-parameter -Wno-unused-function -Wno-overloaded-virtual")
 endif()

and building the DEBUG target. Could not detect any error.

[dfober]

@dsuedholt the problem should be fixed. It was a thread synchronization issue. The fix has been pushed to master-dev branch. Could test?

[dsuedholt]

@dfober just tested with the examples from physicalModeling and my own projects, seems to be working fine now on my end! Thanks for looking into this :)