Segmentation Faults in older package versions of Nuke and Mari

[PumpingPixels]

Hey there, I recently noticed that in older versions of Nuke and Mari, that I occasionally have to use for various reasons, using Qt.py leads to a segmentation fault induced crashes on Linux (CentOS 7.8.2003).

I reproduced this behavior with Nuke 11.1v1 and Mari 4.0v2 which both use Python 2.7.13, while latest Nuke uses 2.7.16, I can't tell for Mari as I have no ongoing maintenance. With these older applications Qt.py works fine up until 1.0.0, while every version beginning with 1.1.0.b1 will lead to a Segmentation fault after running "import Qt" in the host application's Python console.

I have little experience with debugging those kind of issues but I tried to get the stack backtrace with valgrind which gives the following output:

--22824-- memcheck GC: 42216 nodes, 17746 survivors (42.0%)
--22824-- memcheck GC: 42849 new table size (driftup)
--22824-- Reading syms from /home/til/.local/lib/python2.7/site-packages/shiboken2/shiboken2.so
--22824-- ELF section outside all mapped regions
--22824-- Reading syms from /home/til/.local/lib/python2.7/site-packages/shiboken2/shiboken2.so
--22824-- Reading syms from /home/til/.local/lib/python2.7/site-packages/shiboken2/libshiboken2-python2.7.so.5.14
--22824-- ELF section outside all mapped regions
--22824-- Reading syms from /home/til/.local/lib/python2.7/site-packages/shiboken2/libshiboken2-python2.7.so.5.14
==22824== Thread 1:
==22824== Invalid read of size 8
==22824==    at 0x1200BFC9: PyType_Ready (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==    by 0xB3574A6B: add_more_getsets (in /home/til/.local/lib/python2.7/site-packages/shiboken2/libshiboken2-python2.7.so.5.14)
==22824==    by 0xB35775D7: FinishSignatureInitialization (in /home/til/.local/lib/python2.7/site-packages/shiboken2/libshiboken2-python2.7.so.5.14)
==22824==    by 0x1208440F: _PyImport_LoadDynamicModule (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==    by 0x12080CC5: ??? (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==    by 0x12082532: ??? (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==    by 0x12081D45: ??? (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==    by 0x12083FCE: ??? (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==    by 0x1208148D: PyImport_ImportModuleLevel (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==    by 0x1205412D: ??? (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==    by 0x11FE2F8D: PyCFunction_Call (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==    by 0x11F93366: PyObject_Call (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==  Address 0xa8 is not stack'd, malloc'd or (recently) free'd
==22824== 
==22824== Syscall param sendmsg(msg.msg_iov[0]) points to uninitialised byte(s)
==22824==    at 0x453D11: ??? (in /usr/local/Nuke11.1v1/Nuke11.1)
==22824==    by 0x452503: ??? (in /usr/local/Nuke11.1v1/Nuke11.1)
==22824==    by 0x504962F: ??? (in /usr/lib64/libpthread-2.17.so)
==22824==    by 0x1200BFC8: PyType_Ready (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
==22824==  Address 0x29e127b4 is 10,884 bytes inside a block of size 16,384 alloc'd
==22824==    at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==22824==    by 0x44F927: ??? (in /usr/local/Nuke11.1v1/Nuke11.1)
==22824==    by 0x40843F: ??? (in /usr/local/Nuke11.1v1/Nuke11.1)
==22824==    by 0x405CAE: ??? (in /usr/local/Nuke11.1v1/Nuke11.1)
==22824==    by 0x12974661: ??? (in /usr/local/Nuke11.1v1/libnuke-11.1.1.so)
==22824==    by 0x67FD417: ??? (in /usr/local/Nuke11.1v1/libstudio-11.1.1.so)
==22824==    by 0x6BE3CDF: ??? (in /usr/local/Nuke11.1v1/libstudio-11.1.1.so)
==22824==    by 0x4057E3: ??? (in /usr/local/Nuke11.1v1/Nuke11.1)
==22824==    by 0x404EB6: ??? (in /usr/local/Nuke11.1v1/Nuke11.1)
==22824==    by 0x5AA5554: (below main) (in /usr/lib64/libc-2.17.so)
==22824== 
==22824== 
==22824== Process terminating with default action of signal 11 (SIGSEGV)
==22824==    at 0x5B7BBF9: syscall (in /usr/lib64/libc-2.17.so)
==22824==    by 0x4528BB: ??? (in /usr/local/Nuke11.1v1/Nuke11.1)
==22824==    by 0x504962F: ??? (in /usr/lib64/libpthread-2.17.so)
==22824==    by 0x1200BFC8: PyType_Ready (in /usr/local/Nuke11.1v1/libpython2.7.so.1.0)
--22824-- Discarding syms at 0x2a7f01b0-0x2a7f7501 in /usr/lib64/libnss_files-2.17.so due to munmap()
--22824-- Discarding syms at 0x9c8f30a0-0x9c8f69ef in /usr/lib64/libnss_dns-2.17.so due to munmap()
==22824== 
==22824== HEAP SUMMARY:
==22824==     in use at exit: 130,991,708 bytes in 200,666 blocks
==22824==   total heap usage: 1,868,912 allocs, 1,668,246 frees, 11,318,087,303 bytes allocated
==22824== 
==22824== Searching for pointers to 200,666 not-freed blocks
==22824== Checked 1,143,501,624 bytes
==22824== 
==22824== LEAK SUMMARY:
==22824==    definitely lost: 18,100 bytes in 507 blocks
==22824==    indirectly lost: 86,706 bytes in 542 blocks
==22824==      possibly lost: 5,445,045 bytes in 14,166 blocks
==22824==    still reachable: 125,441,857 bytes in 185,451 blocks
==22824==                       of which reachable via heuristic:
==22824==                         stdstring          : 828,692 bytes in 17,422 blocks
==22824==                         length64           : 91,392 bytes in 216 blocks
==22824==                         newarray           : 5,376 bytes in 135 blocks
==22824==                         multipleinheritance: 246,712 bytes in 184 blocks
==22824==         suppressed: 0 bytes in 0 blocks
==22824== Rerun with --leak-check=full to see details of leaked memory
==22824== 
==22824== Use --track-origins=yes to see where uninitialised values come from
==22824== ERROR SUMMARY: 37804 errors from 477 contexts (suppressed: 11 from 1)

Which is then followed by a very long list of "1 errors in context x of 477" and mostly redundant information. It seems that shiboken2 is causing the problem, importing it directly will crash Nuke 11.1v1 as well. Unfortunately, I have little experience how to further investigate this issue from here. Any help would be appreciated.

Thanks and stay healthy, Til

[PumpingPixels]

Never mind, easy solution was just uninstalling the standalone shiboken2 package. Seems to work fine now across all versions of host applications.

[mottosso]

Glad to hear you found a solution, you are most likely spot on with your standalone shiboken2 package. It's likely an issue with PySide2 being built with one compiler, version and set of flags, whereas your standalone shiboken2 package was built differently. That's bound to cause these kinds of issues.

If you for did want your own shiboken2 package, you would likely need to build it for each DCC individually, following the compiler instructions for each of them. Maya for example has very specific requirements on each platform and version of Maya.