JuliaPy / PythonCall.jl

Python and Julia in harmony.
https://juliapy.github.io/PythonCall.jl/stable/
MIT License
720 stars 61 forks source link

`Main.seval` fails to strip whitespace and therefore throws when it shouldn't. #379

Closed LilithHafner closed 8 months ago

LilithHafner commented 8 months ago

Affects: JuliaCall

>>> from juliacall import Main
>>> Main.seval("""
... function f()
...     0
... end
... """)
Julia: f (generic function with 1 method)
>>> if True:
...     Main.seval("""
...     function g()
...         1
...     end
...     """)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/x/.julia/packages/PythonCall/qTEA1/src/jlwrap/module.jl", line 25, in seval
    return self._jl_callmethod($(pyjl_methodnum(pyjlmodule_seval)), expr)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
juliacall.JuliaError: Base.Meta.ParseError("extra token after end of expression")
Stacktrace:
 [1] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:272
 [2] parse
   @ ./meta.jl:266 [inlined]
 [3] pyjlmodule_seval(self::Module, expr::Py)
   @ PythonCall ~/.julia/packages/PythonCall/qTEA1/src/jlwrap/module.jl:13
 [4] _pyjl_callmethod(f::Any, self_::Ptr{PythonCall.C.PyObject}, args_::Ptr{PythonCall.C.PyObject}, nargs::Int64)
   @ PythonCall ~/.julia/packages/PythonCall/qTEA1/src/jlwrap/base.jl:62
 [5] _pyjl_callmethod(o::Ptr{PythonCall.C.PyObject}, args::Ptr{PythonCall.C.PyObject})
   @ PythonCall.C ~/.julia/packages/PythonCall/qTEA1/src/cpython/jlwrap.jl:47

The issue is that Python's multiline string parsing is different from Julia's and does not strip trailing spaces as much, so the strict behavior of Julia's Meta.parse does not make sense for strings that were written using Python's multiline string syntax.

I think PyCall doesn't have this issue, but I'm not positive.

I think that a good solution would be to strip inputs before parsing. It's hard to imagine a case when someone would want code with a leading or trailing space to throw a parse error.

cjdoris commented 8 months ago

Having played around with various combinations of whitespace, this issue only occurs if there is a newline followed by any other character after the expression.

julia> m = Py(Main)
Python: Julia: Main

julia> m.seval("1")
Python: 1

julia> m.seval("1 ")
Python: 1

julia> m.seval("1  ")
Python: 1

julia> m.seval("1\n")
Python: 1

julia> m.seval("1\n ")
ERROR: Python: Julia: Base.Meta.ParseError("extra token after end of expression")

The documentation for Meta.parse says "An error is thrown if there are additional characters after the first expression." Presumably any whitespace before and any whitespace after up to a newline is being treated as part of the expression - which I suppose makes sense because newlines delimit expressions in Julia, but other whitespace does not.