search

JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.47k stars 187 forks source link

Support linking to Pyston rather than CPython #284

Open PallHaraldsson opened 8 years ago

PallHaraldsson commented 8 years ago

I'm not putting any pressure on supporting Pyston [or any other JIT Python], just think of this as a FYI on the interesting thing I discovered; the read on e.g. ref counting vs. former full GC.

[It seems Numba is supported now: https://github.com/stevengj/PyCall.jl/issues/220

I wander if other than Numba JIT, such as Pyston, or more than one JIT Python implementation, is a good idea to support.]

https://blog.pyston.org/2016/05/25/pyston-0-5-released/ "Pyston is now using reference counting. [..] We used a GC initially to try to get more performance. But applying a tracing GC to a refcounting C API, such as the one that Python has, is risky and comes with many performance pitfalls. And most challengingly, Pyston wants to support the large amount of code that has been written that relies on the special properties that refcounting provides (predictable immediate destruction). We found that we had to go to greater and greater lengths to support these programs [..] We are very happy to announce: we can run NumPy, unmodified."

stevengj commented 8 years ago

If (and only if) they support the C API, we should be able to call them.

PallHaraldsson commented 8 years ago

As I said, this is only a FYI, PyCall is awesome with what you've [all] done already, without Pyston support. It's not high priority for me [or anyone?] to support faster Python. People go to Julia for the speed [but not only, I advocate Julia also a better language, without the Hoare's "billion dollar problem", null pointers].

[Julia fixed the "two language problem for speed, as I see it, using Python with, is a (temporary?) crutch to get libraries, for non-speed-critical functionality.]

If anyone likes fast Python with fast Julia:

It seems (I do not really know more than what I'm quoting from the link I gave) they do support the C API (or at least a subset, but want to support all?):

"There are quite a few non-refcounting features that made it into this release as well:

Signal handling Frame introspection of exited frames Generator cleanup Support for more C API functions, such as custom tracebacks

[..]

Why did we do this? In short, because CPython (the main Python implementation) uses refcounting. We used a GC initially to try to get more performance. But applying a tracing GC to a refcounting C API, such as the one that Python has, is risky and comes with many performance pitfalls. And most challengingly, Pyston wants to support the large amount of code that has been written that relies on the special properties that refcounting provides (predictable immediate destruction)."

stevengj commented 8 years ago

If they support most of the C API, and support embedding, we should be able to support them without too much trouble. It's not urgent until people are actually using Pyston for libraries they want to call from Julia. e.g. if Anaconda ever supports Pyston. Until then it won't be worth the trouble.