JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.47k stars 190 forks source link

`deepcopy(::PyObject)` can yield segfault #757

Open tkf opened 4 years ago

tkf commented 4 years ago

I think we need to overload deepcopy_internal(::PyObject, ...) and increment the refcount in it?

julia> using PyCall

julia> a = pycall(pyimport("numpy").zeros, PyObject, 10000)
PyObject array([0., 0., 0., ..., 0., 0., 0.])

julia> pycall(pyimport("sys").getrefcount, Int, a)
2

julia> b = deepcopy(a)
PyObject array([0., 0., 0., ..., 0., 0., 0.])

julia> pycall(pyimport("sys").getrefcount, Int, a)
2

julia> pycall(pyimport("sys").getrefcount, Int, b)  # creating b did not incref
2

julia> pycall(pyimport("builtins").id, Int, a)
140671027744560

julia> pycall(pyimport("builtins").id, Int, b)  # just to show that b _is_ the same Python object
140671027744560

julia> a = nothing

julia> GC.gc(); GC.gc(); GC.gc(); GC.gc(); GC.gc(); GC.gc(); GC.gc()

julia> pycall(pyimport("builtins").id, Int, b)
140671027744560

julia> pycall(pyimport("sys").getrefcount, Int, b)  # now `PyPtr(b)` is pointing nowhere
140671027744561

julia> b  # sometime segfault doesn't happen immediately
PyObject ''uiltin_function_or_method'

julia> pyimport("gc").collect()

signal (11): Segmentation fault
in expression starting at REPL[14]:1
pymalloc_alloc at /tmp/python-build.20200119182925.141496/Python-3.7.6/Objects/obmalloc.c:1420 [inlined]
pymalloc_alloc at /tmp/python-build.20200119182925.141496/Python-3.7.6/Objects/obmalloc.c:1386 [inlined]
_PyObject_Malloc at /tmp/python-build.20200119182925.141496/Python-3.7.6/Objects/obmalloc.c:1578
_PyObject_GC_Alloc at /tmp/python-build.20200119182925.141496/Python-3.7.6/Modules/gcmodule.c:1693 [inlined]
_PyObject_GC_Malloc at /tmp/python-build.20200119182925.141496/Python-3.7.6/Modules/gcmodule.c:1715
_PyObject_GC_New at /tmp/python-build.20200119182925.141496/Python-3.7.6/Modules/gcmodule.c:1727
stevengj commented 4 years ago

Sounds right.