Segfault importing cppyy when called from c++

[marc1uk]

OS: CentOS 7 Python: 3.6.8 CPP: g++4.8.5

Still trying to build a framework that calls python from c++. In order to invoke the python scripts I'm using the classic Python.h API functions: Py_Initialize(), PyImport_Import and PyObject_CallObject. However, on PyImport_Import I get a segfault if my python script includes import cppyy. Running the same python script directly with python3 script.py has no issues.

I've again attached a "minimal" reproducer. The PythonScript.cpp caller is a bit complex - unfortunately it's not my code and I'm not familiar enough with it to strip it down much. Effectively it just has 3 functions: Initialise, Execute and Finalise that invoke the corresponding functions within the python script.

Is there any reason this should fail when called in this way? Or is there another, cleaner way to call python scripts from a parent c++ application?

pydebug.zip

[wlav]

For a proper initialize order, see CPyCppyy's API.cxx, which can be used from the public CPyCppyy/API.h header. Since you were earlier saying you were using ROOT, I believe TPython.h provides something similar. (Note eg. that Py_Initialize() is deprecated.)

As for the segfault, most likely cause in cases like this tends to be a mixup of the Python library you link your executable with and the one that ROOT links libcppyy with. Since that's a ROOT invented "feature", try cppyy proper, as it's unlikely to have that same issue.

[marc1uk]

Thanks for the reference. I've taken a look through API.cxx; it seems similar to what we're doing in PythonScript... there are some calls to PyEval_InitThreads that we're missing... i added those, though I'm not sure quite if they're needed (the current code works without them, provided cppyy is not imported). Beyond that the other code doesn't seem relevant for us - setting argv, and getting a dictionary... If there's something that looks incorrect with our code, please do let me know, this is very much over my head.

Back to the segfault, the other thread I created was for a different installation environment - I should have been more explicit here, but there is a ROOT v5.28 install, which I don't believe should have any interference, while cppyy itself is obtained from github - all current master HEADS except for cppyy-backend which is release cppyy-cling-6.25.2.

[wlav]

Remains that your Makefile uses root-config --libs so even with ROOT v5.28, you'll be pulling in the ROOT libs into the same process. Most of cppyy proper has the old ROOT stuff placed in a namespace CppyyLegacy, but certain globals, such as gInterpreter, are still in the global namespaces. So yeah, that'll crash as cppyy is expecting a pointer to gCling but getting a pointer to gCINT.

I've think we've been here before several times now: do not mix cppyy proper and ROOT. I don't know what else to tell you: it can not be done. There is no workaround. It does not work. It will not work. No-one is going to make it work.

[marc1uk]

ok, apologies. My impression was that the problem was the collision of multiple versions of cppyy internals when one had one of the later versions of ROOT where cppyy had been integrated, so I didn't expect the old version to be problematic. Perhaps this could be made clearer in the docs. I'll abandon cppyy.