search

JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.47k stars 187 forks source link

Do something about PYTHONFAULTHANDLER? #884

Open marius311 opened 3 years ago

marius311 commented 3 years ago

It seems at least on Linux, setting the environment variable PYTHONFAULTHANDLER=1 (enabling a Python-builtin package as of recent versions https://docs.python.org/3/library/faulthandler.html) causes segfaults with PyCall. The MWE I've been using is

# script.jl 

using PyCall

Threads.@sync begin
    Threads.@spawn GC.gc(true)
    Threads.@spawn GC.gc(true)
end

Run julia --threads=2 script.jl and its fine, but run PYTHONFAULTHANDLER=1 julia --threads=2 script.jl and segfault. Reproduced across multiple systems. More impactful I would say, even in single-threaded mode, PYTHONFAULTHANDLER=1 will crash the IJulia kernel any time you try to interrupt it. This is the situation I was in, I was on a cluster where the admins enabled this by default since its a really useful Python thing, and just lived without the ability to interrupt IJulia for months without knowing why, which I can attest really sucks.

Maybe it would make sense to unset this variable in PyCall.__init__ and issue a warning that we're currently incompatible? I could submit a PR.

marius311 commented 3 years ago

(another option is of course to fix this, assuming it can be considered a PyCall bug at all, but I'm much less equipped to do that)