bun_python plugin freezes and segfaults

[joy-void-joy]

Following the instructions in the README, I created bunfig.toml and preload.ts. However running the demo file causes bun to freeze and then

362550 segmentation fault (core dumped)  bun run index.ts

The non-plugin version works fine (For now, I am making do with it and with a slightly bulkier .d.ts file)

[codehz]

I think it is some bug in bun itself, the plugin system has some bug (especially for return object one)... (I suggest use debug version of bun to get verbose error info, and report it to oven/bun)

[heri16]

@codehz

I seem to also be getting a segfault when python code tries to callback into Javascript code, or return values to Javascript code.

Seems like it is related to bun_python missing these important calls:

c.PyEval_InitThreads() or c.PyGILState_Ensure()

Related discussion: https://stackoverflow.com/questions/4866701/python-pygilstate-ensure-release-causes-segfault-while-returning-to-c-from-p

My environment is bunjs on Ubuntu Jammy x86-64. Also happens sporadically on another machine which is a MacBook Pro M1.

[heri16]

// initialize the Python interpreter
Py_Initialize();

PyEval_InitThreads();

/* Swap out and return current thread state and release the GIL */
PyThreadState tstate = PyEval_SaveThread();

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

PyRun_SimpleString("import random\n");

PyGILState_Release(gstate); 

From: https://bugs.python.org/msg57619

You can find documentation for these functions in the Python C API documentation. Here are the relevant sections for each function:

• Py_Initialize(): Initializes the Python interpreter. You can find its documentation here.

• PyEval_InitThreads(): Initializes the threading subsystem in Python. You can find its documentation here.

• PyEval_SaveThread(): Swaps out and returns the current thread state, releasing the GIL. You can find its documentation here.

• PyGILState_Ensure(): Ensures that the calling thread has acquired the GIL. You can find its documentation here.

• PyRun_SimpleString(): Executes a Python string in the context of the current thread. You can find its documentation here.

• PyGILState_Release(): Releases the GIL, potentially allowing other threads to acquire it. You can find its documentation here.

These functions are part of the Python C API and are used when embedding Python into a C/C++ application.

[heri16]

Submitted PR #2

[codehz]

I just noticed that there may be another cause of the crash... not just the GIL issue

[heri16]

Care to elaborate?

[codehz]

@heri16 has been fixed, related to gc https://github.com/codehz/bun_python/commit/e28c1824505eba9c90fcd6864e32fc6a86518bdd#diff-08b76b1156c14219681dc54823e0b39fd0e2e719ead10a5ebda8aeeb312fdf4aR224-R228

[heri16]

Seems like segfaults are still happening for Callbacks from. Python to JS, in cases of more complex python code.

[heri16]

Found the problem, it is due to python.callback() not returning a Callback that holds a reference to the PyObject, causing GC to happen too early on the PyObject, before the Callback.destroy() is called.

This (line 506) fixed all remaining segfaults/crashes for me.

[heri16]

Submitted PR #4