JuliaPy / PythonCall.jl

Python and Julia in harmony.
https://juliapy.github.io/PythonCall.jl/stable/
MIT License
776 stars 63 forks source link

Working with type instabilities, coming from PyJulia #441

Open MilesCranmer opened 8 months ago

MilesCranmer commented 8 months ago

@cjdoris Is there an automatic way to force type conversions when passing Python objects to methods? Or, in other words, is there a way to automatically convert Python arguments to their Julia counterparts?

For example I am running into this issue right now when I pass a list of integers:

>>> from juliacall import Main as jl
>>> jl.seval("f(x) = (@show typeof(x); nothing)")
Julia: f (generic function with 1 method)
>>> jl.f([1, 2, 3])
typeof(x) = PyList{Any}

which causes some issues as now f is unaware of the element type of this vector.

However, in PyJulia, arguments seem to somehow get converted automatically:

>>> from julia import Main as jl
>>> jl.eval("f(x) = (@show typeof(x); nothing)")
<PyCall.jlwrap f>
>>> jl.f([1, 2, 3])
typeof(x) = Vector{Int64}

Is there a way to get this same behavior in PythonCall?


Possibly related to #439

MilesCranmer commented 8 months ago

Okay it seems like this was all explained here: https://juliapy.github.io/PythonCall.jl/stable/pythoncall/#Conversion-between-Julia-and-Python

and is the expected behavior. So I simply need to run a pyconvert(Vector, ...) for any mutable vector inputs when calling.

It would be great if this could be the default behavior instead of PyList{Any} - I think it is more in-line with the user expectations to have it converted to a Vector{Float32}, and of course this is what PyJulia does, so it would maintain compatibility with packages that are switching.

cjdoris commented 8 months ago

See also https://juliapy.github.io/PythonCall.jl/stable/juliacall-reference/#juliacall.convert, which allows you to convert the object Python-side before passing it into the Julia function.

cjdoris commented 8 months ago

Thinking more hard about the default conversion behaviour of containers is on the roadmap for PythonCall/JuliaCall v1.0. The original intent was to use wrapper types for speed and mutability, but this forces the eltype to be Any which turns out to not be useful. So 1.0 may indeed convert lists to vectors.

MilesCranmer commented 8 months ago

I think converting lists to vectors by default would be the better option for a variety of reasons. The inexperienced users are the ones most likely to shoot themselves in the foot by passing mutable objects to a Julia call, and also the least aware of the issues caused by type instability.

Experienced users will find a way to mutate lists if they need to so I think the default interface should just create a vector.