JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.47k stars 187 forks source link

strides is deprecated #555

Open cstjean opened 6 years ago

cstjean commented 6 years ago

On 0.7, I can pass a reshaped range to PyCall

julia> np.sin(reshape(1:4, 2,2))
┌ Warning: The default `strides(a::AbstractArray)` implementation is deprecated for general arrays.
│ Specialize `strides(::Base.ReshapedArray)` if `Base.ReshapedArray` indeed uses a strided representation in memory.
│ Warning: inappropriately implementing this method for an array type that does not use strided
│ storage may lead to incorrect results or segfaults.
│   caller = stride at abstractarray.jl:350 [inlined]
└ @ Core ./abstractarray.jl:350
┌ Warning: The default `strides(a::AbstractArray)` implementation is deprecated for general arrays.
│ Specialize `strides(::Base.ReshapedArray)` if `Base.ReshapedArray` indeed uses a strided representation in memory.
│ Warning: inappropriately implementing this method for an array type that does not use strided
│ storage may lead to incorrect results or segfaults.
│   caller = stride at abstractarray.jl:350 [inlined]
└ @ Core ./abstractarray.jl:350
2×2 Array{Float64,2}:
 0.841471   0.14112 
 0.909297  -0.756802

On 1.0, it's

ERROR: MethodError: no method matching strides(::Base.ReshapedArray{Int64,2,UnitRange{Int64},Tuple{}})
Closest candidates are:
  strides(::SubArray) at subarray.jl:251
  strides(::Base.CodeUnits) at strings/basic.jl:696
  strides(::PermutedDimsArray{T,N,perm,iperm,AA} where AA<:AbstractArray where iperm) where {T, N, perm} at permuteddimsarray.jl:62
  ...
Stacktrace:
 [1] stride(::Base.ReshapedArray{Int64,2,UnitRange{Int64},Tuple{}}, ::Int64) at ./abstractarray.jl:342
 [2] array2py(::Base.ReshapedArray{Int64,2,UnitRange{Int64},Tuple{}}, ::Int64, ::Int64) at /home/cst-jean/.julia/packages/PyCall/rUul9/src/conversions.jl:305
 [3] array2py(::Base.ReshapedArray{Int64,2,UnitRange{Int64},Tuple{}}) at /home/cst-jean/.julia/packages/PyCall/rUul9/src/conversions.jl:325
 [4] Type at /home/cst-jean/.julia/packages/PyCall/rUul9/src/conversions.jl:327 [inlined]
 [5] macro expansion at /home/cst-jean/.julia/packages/PyCall/rUul9/src/exception.jl:84 [inlined]
 [6] _pycall!(::PyObject, ::PyObject, ::Tuple{Base.ReshapedArray{Int64,2,UnitRange{Int64},Tuple{}}}, ::Int64, ::Ptr{Nothing}) at /home/cst-jean/.julia/packages/PyCall/rUul9/src/pyfncall.jl:21
 [7] #call#89 at /home/cst-jean/.julia/packages/PyCall/rUul9/src/pyfncall.jl:11 [inlined]
 [8] (::PyObject)(::Base.ReshapedArray{Int64,2,UnitRange{Int64},Tuple{}}) at /home/cst-jean/.julia/packages/PyCall/rUul9/src/pyfncall.jl:89

Is that a PyCall issue, or a base issue?

stevengj commented 6 years ago

I think array2py needs to be rewritten based on cartesian indexing.

tkf commented 6 years ago

Why don't make the conversion lazy (unless the arrays is representable as a "native" numpy array)?

Here is my experiment on the "lazy" approach:

The array interface __array_interface__ can also be implemented as a lazy property so that numpy.asarray can use Julia function copy internally (but only when necessary).

Datseris commented 6 years ago

Ping here; this is also (I think) the same problem when passing a transpose:

ERROR: LoadError: MethodError: no method matching strides(::LinearAlgebra.Adjoint{Float64,Array{Float64,2}})
Closest candidates are:
  strides(::SubArray) at subarray.jl:251
  strides(::Base.CodeUnits) at strings/basic.jl:696
  strides(::PermutedDimsArray{T,N,perm,iperm,AA} where AA<:AbstractArray where iperm) where {T, N, perm} at permuteddimsarray.jl:62
  ...
Stacktrace:
 [1] stride(::LinearAlgebra.Adjoint{Float64,Array{Float64,2}}, ::Int64) at .\abstractarray.jl:342
 [2] array2py(::LinearAlgebra.Adjoint{Float64,Array{Float64,2}}, ::Int64, ::Int64) at C:\Users\datseris\.julia\packages\PyCall\rUul9\src\conversions.jl:305
 [3] array2py at C:\Users\datseris\.julia\packages\PyCall\rUul9\src\conversions.jl:325 [inlined]
 [4] Type at C:\Users\datseris\.julia\packages\PyCall\rUul9\src\conversions.jl:327 [inlined]
 [5] macro expansion at C:\Users\datseris\.julia\packages\PyCall\rUul9\src\exception.jl:84 [inlined]
 [6] _pycall!(::PyCall.PyObject, ::PyCall.PyObject, ::Tuple{LinearAlgebra.Adjoint{Float64,Array{Float64,2}}}, ::Int64, ::PyCall.PyObject) at C:\Users\datseris\.julia\packages\PyCall\rUul9\src\pyfncall.jl:21
 [7] _pycall!(::PyCall.PyObject, ::PyCall.PyObject, ::Tuple{LinearAlgebra.Adjoint{Float64,Array{Float64,2}}}, ::Base.Iterators.Pairs{Symbol,Any,NTuple{4,Symbol},NamedTuple{(:origin, :extent, :cmap, :aspect),Tuple{String,Array{Float64,1},String,String}}}) at C:\Users\datseris\.julia\packages\PyCall\rUul9\src\pyfncall.jl:11
 [8] #pycall#88 at C:\Users\datseris\.julia\packages\PyCall\rUul9\src\pyfncall.jl:86 [inlined]
 [9] #pycall at .\none:0 [inlined]
 [10] #imshow#67(::Base.Iterators.Pairs{Symbol,Any,NTuple{4,Symbol},NamedTuple{(:origin, :extent, :cmap, :aspect),Tuple{String,Array{Float64,1},String,String}}}, ::Function, ::LinearAlgebra.Adjoint{Float64,Array{Float64,2}}) at C:\Users\datseris\.julia\packages\PyPlot\fZuOQ\src\PyPlot.jl:179
 [11] (::getfield(PyPlot, Symbol("#kw##imshow")))(::NamedTuple{(:origin, :extent, :cmap, :aspect),Tuple{String,Array{Float64,1},String,String}}, ::typeof(imshow), ::LinearAlgebra.Adjoint{Float64,Array{Float64,2}}) at .\none:0

when trying to imshow this: some_matrix'

Alexander-Barth commented 5 years ago

This avoids the problem, by creating an additional copy of the data.

using PyCall: PyObject
PyObject(x::Adjoint) = PyObject(copy(x))
PyObject(x::Transpose) = PyObject(copy(x))

Can this be added to PyCall? It would also solve https://github.com/JuliaPy/PyPlot.jl/issues/380 .