JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.47k stars 190 forks source link

Conversion of read-only numpy arrays #715

Open marcoct opened 5 years ago

marcoct commented 5 years ago

Currently, a read-only numpy array is not converted into either an Array or a PyArray automatically:

using PyCall

py"""
import numpy

def get_read_only_narray():
    x = numpy.zeros(10)
    x.flags.writeable = False
    return x
"""

arr = py"get_read_only_narray"()
@assert isa(arr, PyObject)

Attempting to convert manually into a PyArray with PyArray(arr) gives an error "buffer source array is read-only", which makes sense because https://github.com/JuliaPy/PyCall.jl/blob/master/src/pyarray.jl#L16 seems to be assuming the buffer is writeable (PyBUF_ND_STRIDED implies PyBUF_WRITABLE). This seems like it could be considered a bug in PyArray constructor, since PyArray does check for the readonly flag later in the constructor: https://github.com/JuliaPy/PyCall.jl/blob/master/src/pyarray.jl#L22

I worked around this:

# workaround
function convert_it(read_only_numpy_arr::PyObject)

    # See https://github.com/JuliaPy/PyCall.jl/blob/master/src/pyarray.jl#L14-L24

    # instead of PyBUF_ND_STRIDED =  Cint(PyBUF_WRITABLE | PyBUF_FORMAT | PyBUF_ND | PyBUF_STRIDES)
    # See https://github.com/JuliaPy/PyCall.jl/blob/master/src/pybuffer.jl#L113
    pybuf = PyBuffer(read_only_numpy_arr, PyCall.PyBUF_FORMAT | PyCall.PyBUF_ND | PyCall.PyBUF_STRIDES)

    T, native_byteorder = PyCall.array_format(pybuf)
    sz = size(pybuf)
    strd = PyCall.strides(pybuf)
    length(strd) == 0 && (sz = ())
    N = length(sz)
    isreadonly = pybuf.buf.readonly==1
    info = PyCall.PyArray_Info{T,N}(native_byteorder, sz, strd, pybuf.buf.buf, isreadonly, pybuf)

    # See https://github.com/JuliaPy/PyCall.jl/blob/master/src/pyarray.jl#L123-L126
    PyCall.PyArray{T,N}(read_only_numpy_arr, info)
end

arr2 = convert_it(arr) # arr2 is a PyArray
@assert isa(arr2, PyArray)
@assert size(arr2) == (10,)
println(arr2)

Based on the above, it seems possible to fix the PyArray constructor to handle this case. Separately, it would be even more useful to automatically convert read-only numpy arrays into an Array (and do a copy).

SebastianM-C commented 4 years ago

I also encountered this problem and the above code fixed it for me. It would be great if the fix could be incorporated.