search

JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.47k stars 189 forks source link

handling arrays with missing values (with possible solution) #616

Open Alexander-Barth opened 5 years ago

Alexander-Barth commented 5 years ago

I would like to plot (with PyPlot) an array containing missing values:

using PyPlot, Missings
d = [missing 1; 2 3] 
pcolor(d)

Unfortunately, I get the following error with PyCall v1.18.5, PyPlot v2.6.3 and Julia v1.0.1:

PyError ($(Expr(:escape, :(ccall(#= /home/abarth/.julia/packages/PyCall/0jMpb/src/pyfncall.jl:44 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw))))) <class 'TypeError'>
TypeError('unorderable types: PyCall.jlwrap() <= int()',)
  File "/home/abarth/.local/lib/python3.5/site-packages/matplotlib/pyplot.py", line 2761, in pcolor
    **kwargs)
  File "/home/abarth/.local/lib/python3.5/site-packages/matplotlib/__init__.py", line 1810, in inner
    return func(ax, *args, **kwargs)
  File "/home/abarth/.local/lib/python3.5/site-packages/matplotlib/axes/_axes.py", line 5785, in pcolor
    collection.autoscale_None()
  File "/home/abarth/.local/lib/python3.5/site-packages/matplotlib/cm.py", line 375, in autoscale_None
    self.norm.autoscale_None(self._A)
  File "/home/abarth/.local/lib/python3.5/site-packages/matplotlib/colors.py", line 988, in autoscale_None
    self.vmin = A.min()
  File "/home/abarth/.local/lib/python3.5/site-packages/numpy/core/_methods.py", line 32, in _amin
    return umr_minimum(a, axis, None, out, keepdims, initial)

Stacktrace:
 [1] pyerr_check at /home/abarth/.julia/packages/PyCall/0jMpb/src/exception.jl:60 [inlined]
 [2] pyerr_check at /home/abarth/.julia/packages/PyCall/0jMpb/src/exception.jl:64 [inlined]
 [3] macro expansion at /home/abarth/.julia/packages/PyCall/0jMpb/src/exception.jl:84 [inlined]
 [4] __pycall!(::PyCall.PyObject, ::Ptr{PyCall.PyObject_struct}, ::PyCall.PyObject, ::Ptr{Nothing}) at /home/abarth/.julia/packages/PyCall/0jMpb/src/pyfncall.jl:44
 [5] _pycall!(::PyCall.PyObject, ::PyCall.PyObject, ::Tuple{Array{Union{Missing, Int64},2}}, ::Int64, ::Ptr{Nothing}) at /home/abarth/.julia/packages/PyCall/0jMpb/src/pyfncall.jl:22
 [6] #pycall#88 at /home/abarth/.julia/packages/PyCall/0jMpb/src/pyfncall.jl:11 [inlined]
 [7] pycall at /home/abarth/.julia/packages/PyCall/0jMpb/src/pyfncall.jl:86 [inlined]
 [8] #pcolor#81(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Array{Union{Missing, Int64},2}) at /home/abarth/.julia/packages/PyPlot/fZuOQ/src/PyPlot.jl:179
 [9] pcolor(::Array{Union{Missing, Int64},2}) at /home/abarth/.julia/packages/PyPlot/fZuOQ/src/PyPlot.jl:176
 [10] top-level scope at In[1]:3

However, if declare the following convertion rule in PyCall, the plotting works:

using PyCall
using PyCall: PyObject
function PyObject(a::Array{Union{T,Missing},N}) where {T,N}
  numpy_ma = pyimport("numpy")["ma"]
  pycall(numpy_ma["array"], Any, coalesce.(a,zero(T)), mask=ismissing.(a))
end
pcolor(d)

The function PyCall creates essentially a numpy masked array. All missing values are replaced by zero and a mask is given to indicate which values are missing.

Can this be added to PyCall?

Alexander-Barth commented 5 years ago

Would it be useful to make a pull request adding this function?

Fixing deprecated function calls, this is what I currently use:

using PyCall
using PyCall: PyObject

# allow for plotting with missing values
function PyObject(a::Array{Union{T,Missing},N}) where {T,N}
    numpy_ma = PyCall.pyimport("numpy").ma
    pycall(numpy_ma.array, Any, coalesce.(a,zero(T)), mask=ismissing.(a))
end