steve-s
commented
2 years ago Fundamentally no JIT can "see though" a native code of an extension, so it cannot optimize it and it cannot inline such code into the surrounding Python code to optimized the whole thing as one compilation unit.
GraalPython has a mode where it does not execute the actual native code, but LLVM bitcode inside the same JIT as the Python code. So there we can theoretically get on par or close. I changed your benchmark a bit, because GraalPython needs bit longer to warm up, so I excluded first few iterations. Here are the results of GraalPython:
- list: 0.00012 s
- numpy: 0.34357 s
- piconumpy HPy (universal mode): 0.00215 s
We do not get on par, because there is also one more issue of the API itself. Consider the underlying C function that gets called in the loop:
HPyDef_SLOT(Array_item, Array_item_impl, HPy_sq_item)
HPy Array_item_impl(HPyContext *ctx, HPy h_arr, HPy_ssize_t index) {
ArrayObject *arr = ArrayObject_AsStruct(ctx, h_arr);
if (index < 0 || index >= arr->size) {
HPyErr_SetString(ctx, ctx->h_IndexError, "index out of range");
return HPy_NULL;
}
HPy item = HPyFloat_FromDouble(ctx, arr->data[index]);
return item;
};The issue is that:
HPyErr_SetString(ctx, ctx->h_IndexError, "index out of range");really needs to create the exception unlike if it is pure Python code with a Python list, where we can optimize the exception away. In theory we should be able to optimize it away across the language boundary if both languages are compiled with the same JIT, but in practice it turns out to be much more difficult. Among other things we'd need to analyze the C code between HPyErr_SetString and return HPy_NULL to see that it's actually not doing anything with the exception.
When I hacked GraalPython to not create the full Python level exception in this case (which is not correct in general), the performance gets closer:
- piconumpy HPy (with IndexError hack): 0.00013 s
I think that even for Pypy it would be useful if it knew that it actually does not have to create the exception. If we agree this is really important use case, we could, for example, provide some token that, if returned, would indicate that the HPy function wishes to exit with an exception of type IndexError, so the code would be:
HPyDef_SLOT(Array_item, Array_item_impl, HPy_sq_item)
HPy Array_item_impl(HPyContext *ctx, HPy h_arr, HPy_ssize_t index) {
ArrayObject *arr = ArrayObject_AsStruct(ctx, h_arr);
if (index < 0 || index >= arr->size) {
return HPy_ErrorResult_IndexError;
}
HPy item = HPyFloat_FromDouble(ctx, arr->data[index]);
return item;
};
hodgestar
paugier
mattip
This is a long term question/issue related to the piconumpy experiment and the performance results that I get with HPy and PyPy. These results are now so and so.
I thought piconumpy could be used to demonstrate the potential of HPy for numerical computing. With good results on small examples related to numerical computing with Python, I'm convinced that a lot of funding can easily be obtained for HPy. But we would need impressive results.
For numerical computing with Python, we lack in pure Python homogeneous ndarrays of native types and of user defined classes. Numpy provides homogeneous ndarrays of native types and inhomogeneous ndarrays of Python objects. Great! But Numpy is written in C and thus low level code using Numpy can't be accelerated with a JIT accelerating pure Python code. As soon as the C/Python border is crossed too much, it becomes super slow for any Python implementations, even with a good JIT.
The available solutions are based on moving the C/Python border outside of the hot code by reimplementing Numpy/Python (as in Pythran, Numba or NumPyPy).
A more general solution without reimplementation would be a game changer. I'd like to understand if HPy could go in this direction.
This very simple microbenchmark on just summing floats is sufficient to explain what I mean here. With Piconumpy now ported to HPy, we can start to see what it could give. The function benchmarked is just:
The results can be summarized as follow. I normalized by the time obtained with Julia.
With PyPy3.7
With CPython
Comments on the results
PyPy is fast with lists (1.3 * Julia!) and ~ as fast for a purepython piconumpy array based on a list ("piconumpy.purepy", zero cost abstraction!)
Numpy and _piconumpy_cpython_capi are both much slower with PyPy than with Cpython. We can guess that the Numpy port to HPy would fix that.
piconumpy_hpy is a bit faster with PyPy (19 Julia) than with CPython (40 Julia), however, we see that PyPy does not strongly accelerate piconumpy_hpy (19 Julia = 10 piconumpy_list).
I'm not familiar enough with PyPy traces to understand how the JIT handles the hot loop in the case of piconumpy_hpy (the traces are here), but the performance is consistent with a very small acceleration by the JIT. It seems that PyPy JIT does not "understand" the "nature of piconumpy_hpy array" and is not able to use its good characteristics for performance (fixed size, homogeneous in type, aligned native array, ...).
Few questions:
Are there potentials for improvement of acceleration in the case of the piconumpy_hpy array ? How much could we win (keeping the the same HPy code in piconumpy)?
Would there be some needs of an extended API to be able to describe in C array-like types leading in PyPy to results similar to homogeneous lists.