ztane
commented
7 years ago Well, first of all, it is not really possible to orderly recover from a "segfault". The C language doesn't guarantee any such thing; the best thing that could be possibly done is to attempt to dump some python state on segfault, but even it is not guaranteed.
And GIL is not really required for the Python code itself, but all the C code that poke the internals of objects. It is true that if all the C code was replaced by Python code running in a minimal interpreter, things would be easier though - this is the approach taken by Java wherein most of the standard library is written in Java translated into Java byte code, and not as C modules, as CPython currently does.
ghost
Rather than attempting to modify the whole of python to work multithreaded, perhaps a more incremental approach could be taken. What I envision is this: having a
with nogil:statement that simply unlocks the GIL, and then having python move all of the variables coming into the context to a special "nogil" section that the interpreter with GIL would not be able to access. Python would then just be responsible for converting any segfaults into Python SystemExit exceptions rather than crashing hard.The main advantage of this is that it can be implemented now because the python code would be responsible for managing the locks rather than Python itself. This would also support iterative design. In addition, single-threaded performance would suffer minimally if nogil is not invoked and the same interpreter can run existing code without nogil.
The main disadvantage is that it would make nogil code more complex. But solving problems in Python is easier and faster than solving problems in C.
Reasons you should listen to me: none. I will not have time to devote to this as I am already working on other projects. But I would like to see a nogil implementation in Python and if I can give feedback toward that end, then I will.