Hello, HPy | HPy

[utterances-bot]

Hello, HPy | HPy

https://hpyproject.org/blog/posts/2021/03/hello-hpy/

[mattip]

Can't wait to hear more.

[magicl]

This is great. Looking forward to a more compatible PyPy due to this work, so we can actually start using it :) . Keep it up!

[lostmsu]

Is this going to cover embedding scenarios as well as extension development? CPython API does that.

[antocuni]

@lostmsu From the API point of view, there is nothing in HPy which would prevent to use it for embedding, so eventually we will also support that. However, the first priority for now are extensions.

[lostmsu]

@antocuni embedding into non-C languages precludes using C macros to define new objects.

I think if API is not designed with that in mind, it will have the same problems as CPython has for embedding.

[antocuni]

@lostmsu macros are not required to write HPy extensions. Unfortunately HPy is a bit under-documented now and it's not easy to find info about this, but the various macros like HPyDef_METH & co. are there just to simplify the life of the users. It is possible to do things manually and define HPyDef by hand.

This is an example taken from ultrajson-hpy:

// ujson.c does something a bit weird: it defines two Python-level methods
// ("encode" and "dumps") for the same C-level function
// ("objToJSON_impl"). HPyDef_METH does not support this use case, but we can
// define our HPyDef by hand: HPyDef_METH is just a convenience macro and the
// structure and fields of HPyDef is part of the publich API
HPyDef objToJSON_encode = {
   .kind = HPyDef_Kind_Meth,
   .meth = {
       .name = "encode",
       .impl = objToJSON_impl,
       .cpy_trampoline = objToJSON_trampoline,
       .signature = HPyFunc_KEYWORDS,
       .doc = ("Converts arbitrary object recursively into JSON. "
               ENCODER_HELP_TEXT),
   }
};

[aneeshdurg]

Great article!

Minor note: should .m_name = "hello_old", be .m_name = "hello_new, instead?

[antocuni]

@aneeshdurg you are correct, thanks for reporting. Fixed by 935e8f6, should appear online soon

[PimvanderEijk]

Very interesting. When do you expect the first official release?

[antocuni]

@PimvanderEijk it depends on how you define "official".

We recently agreed that it's a good idea to start tagging some git versions with release numbers: this will simplify the interactions with hpy, pypy and grallpython. Once we do that, we might or might not decide to also publish the releases on PyPI, but this should be a minor thing as anyone who is brave enough to try hpy at this stage should not have problems at installing it from git.

However, note that even if we start to roll out releases, we are not making any commitment of compatibility between them at this stage. The API is still subject to change completely between one release and the next, see e.g. https://github.com/hpyproject/hpy/pull/182.

If the question is "when do we expect to roll out a release on which people can rely to write and distribute their own extensions", I think it will be several months from now.