python / cpython

The Python programming language
https://www.python.org
Other
63.77k stars 30.54k forks source link

Add `PyMem_Raw{New,Resize}` convenience macros #127415

Open picnixz opened 1 day ago

picnixz commented 1 day ago

Feature or enhancement

Proposal:

We have convenience macros for PyMem_New and PyMem_Resize to allocate memory for n objects of the given type without checking if n * sizeof(T) would overflow but we don't have any for the raw allocators. I suggest adding those two macros.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

encukou commented 1 day ago

It seems to me that they wouldn't be worth it. PyMem_RawCalloc(sizeof(T), n) isn't that much worse to type than PyMem_RawNew(T, n), and PyMem_Resize isn't very convenient if you want proper error handling.

ZeroIntensity commented 1 day ago

Looking at the PR, I think the biggest difference would be that PyMem_RawNew doesn't zero-out the memory. I personally don't find that useful, but I guess some users that are gung-ho on performance might like it.

picnixz commented 1 day ago

It isn't that much worse but it does use a different allocator function (one being calloc(), the other one being malloc()). In addition, IIRC, calloc() sets the memory to 0 while malloc() leaves area of memory uninitialized. For downstream users, I feel that it's better that they retain the semantics they want as well.

ZeroIntensity commented 1 day ago

I do see some usage, but Petr is less easily convinced. Generally for C API changes, you need to bicker with the C API WG first: https://github.com/capi-workgroup/decisions/issues

picnixz commented 1 day ago

Oh right. I'll move the discussion there (I tend to forget about the C API WG when the feature is small enough but I can try writing a more convincing proposal tomorrow)