hodgestar
commented
1 year ago @paugier Thank you for giving the talk and for writing all of this up. I agree that we could be a lot more upbeat. Much of the documentation was written a couple of years ago when HPy was just starting out. It definitely needs an update to match the current much more usable and mature status of the project.
Would incorporating the manifesto and the What needs to change and why help explain the purpose to HPy users / the broader community or is it still too technical?
paugier
mattip
I worked on a presentation about HPy for the french "Calcul group".
The pdf of the presentation is here.
I wrote some notes about HPy mostly for me to prepare the presentation. When HPy 0.9 will be released and if I feel that it can be useful, I could share another version of this text (a bit more positive), for example on Numpy mailing list.
This version contains some parts that could potentially be taken into account to improve the project, so I'm going to open few issues here to share my (user) point of view.
I start with the section on communication that I just copy here. It could be better to understand my point to read the whole text.
Communication about HPy
Currently, the external communication on HPy is mostly done through the project README, the project website/blog and the project documentation.
Most of these documents target people working with the CPython C API and this technical documentation is IMO remarkably clear and educational. Unfortunately, HPy communication is less good to motivate maintainers of projets and end-users to support the project. There is a very interesting page HPy overview which nicely presents the motivation and goals. However, some high level points of view are IMHO missing:
There are also very few mentions of HPy in Python conferences and social media. Victor Stinner mentions the project in some of his talks, for example Python Performance: Past, Present and Future at EuroPython 2019 and Introducing incompatible changes in Python at PyCon US 2023. However, there won't be any talks centered on HPy at PyCon 2023 (except in the Language Summit).
It is interesting to compare the respective communications of HPy and of the Faster CPython project. The Faster CPython project presents very high ambitions (X5 speedup for CPython in few years) and explains its detailed plan through many channels, for example a talk by Mark Shannon at PyCon 2023. In contrast, HPy is very shy and conservative. It is repeated in the documentation that "HPy is still in the early stages of development" and that "there is still a long road before HPy is usable for the general public". This is partially true, but this is not very positive and motivating. The concrete consequences for the end-users are somehow hidden behind quite technical data.
However, with all the respect that I have for the people working on the Faster CPython project, a successful HPy project (i.e. most popular packages with extensions using HPy) would lead to a deeper and better improvement to the Python ecosystem. In particular, it would bring great speedup for several users, which will be able to use Python implementations really "5 times faster" than Python 3.10 for many real world applications. Having specialized interpreters for different tasks would be much better than one interpreter that still has to support its problematic legacy C API during years. Without improvements of the CPython C API and with the constrain of not degrading performance of extensions using the legacy C API, we start to know that the target "x5 faster" of the Faster CPython project is very ambitious. Thus, a successful HPy would actually help a lot the Faster CPython project.