Private APIs used in Pyodide

[hoodmane]

Here is the list of the private C APIs used by Pyodide. I tried to drop ones that have public equivalents on the main branch, but I may have accidentally kept some.

cc @vstinner

Argument parsing

• _PyArg_CheckPositional -- argument clinic uses it, probably should be public?

Argument parsing for vectorcall, would be nice to have argument clinic variant for these I guess?

• _PyArg_Parser
• _PyArg_ParseStack
• _PyArg_ParseStackAndKeywords

Errors and tracebacks

• _Py_DumpTraceback -- used for fatal errors
• _PyTraceback_Add -- add JS frames to a Python traceback
• _PyErr_FormatFromCause -- Chain exceptions conveniently

Generators

• _PyGen_FetchStopIterationValue -- used to consume generators
• _PyGen_SetStopIterationValue -- used to implement generators

Hashes

• _Py_HashBytes -- use Python utility function for hashing my stuff rather than reimplement
• _PyDict_GetItem_KnownHash -- several lookups in a row on same key

Numbers

• _PyLong_AsByteArray -- For converting a Python int to a JS bigint
• _PyLong_FromByteArray -- For converting a JS bigint to a Python int
• _PyLong_NumBits -- Work out how much space to allocate for the bytearray passed to PyLong_FromByteArray
• _PyNumber_Index -- much more common in internal Python code than PyNumber_Index. PyNumber_Index forces the return type to be exactly PyLong rather than a subtype. We can handle subtypes so we use the underscore version.

Other data structures

• _PySet_Update
• _PyUnicode_EQ

Miscellaneous

• _PyObject_GetMethod -- caching for lookups on Python objects from JavaScript
• _PyObject_NextNotImplemented -- checking if a Python object supports iteration, could be replaced with PyIter_Check

PyID functions

I could easily shim these; Pyodide is single threaded so they are nice.

• _Py_IDENTIFIER
• _PyObject_CallMethodIdNoArgs
• _PyObject_CallMethodIdObjArgs
• _PyObject_CallMethodIdOneArg
• _PyObject_GetAttrId
• _PyObject_SetAttrId
• _PyUnicode_FromId

[vstinner]

_Py_HashBytes -- use Python utility function for hashing my stuff rather than reimplement

The API was approved but not implemented: https://github.com/capi-workgroup/decisions/issues/13

[encukou]

Some of these should have a public alternative. (Maybe already evaluated some of these and found a reason to not use them?)

_PyArg_CheckPositional

Argument Clinic is private itself. But, if you run it with --limited, it'll keep to the limited (and thus public) API. (And the replacement for _PyArg_CheckPositional even hardcodes some things that the function determines at runtime!)

_PyLong_AsByteArray / _PyLong_FromByteArray / _PyLong_NumBits

There is now PyLong_AsNativeBytes/PyLong_FromNativeBytes. The As also serve the _PyLong_NumBits use case (and if you can optimistically give it a small buffer, only huge integers will need two calls)

_PyNumber_Index

You can use tp_as_number->nb_index, and type-check the result yourself. Note that returning int subtypes from nb_index (or __index__) is deprecated.

_PySet_Update

Call PyNumber_InPlaceOr, or tp_as_number->nb_inplace_or. (Assuming you don't mind that subclasses can override this.)

[vstinner]

_PyUnicode_EQ

I created https://github.com/python/cpython/issues/124502 to add PyUnicode_Equal() function.

[vstinner]

_Py_HashBytes -- use Python utility function for hashing my stuff rather than reimplement

Py_HashBuffer() was added to Python 3.14: https://docs.python.org/dev/c-api/hash.html#c.Py_HashBuffer