syntactic sugar: type coercion on pointer assignment

[57e0b051-ea9a-4763-9e60-246f465cb192]

BPO	5149
Nosy	@amauryfa, @abalkin, @meadori

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['ctypes', 'type-feature', '3.10'] title = 'syntactic sugar: type coercion on pointer assignment' updated_at = user = 'https://bugs.python.org/PeterASilva' ``` bugs.python.org fields: ```python activity = actor = 'iritkatriel' assignee = 'none' closed = False closed_date = None closer = None components = ['ctypes'] creation = creator = 'PeterASilva' dependencies = [] files = [] hgrepos = [] issue_num = 5149 keywords = [] message_count = 5.0 messages = ['81123', '141747', '143452', '143657', '143659'] nosy_count = 5.0 nosy_names = ['amaury.forgeotdarc', 'belopolsky', 'PeterASilva', 'meador.inge', 'vladris'] pr_nums = [] priority = 'normal' resolution = None stage = 'needs patch' status = 'open' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue5149' versions = ['Python 3.10'] ```

[57e0b051-ea9a-4763-9e60-246f465cb192]

tough% cat toy.py

from ctypes import *

class foo(Structure):
  _fields_ = [ ( "bar", c_char_p ) ]

foofoo = foo()

babar = create_string_buffer(32)

foofoo.bar = babar

tough% python toy.py
Traceback (most recent call last):
  File "toy.py", line 11, in <module>
    foofoo.bar = babar
TypeError: incompatible types, c_char_Array_32 instance instead of
c_char_p instance
tough%

would be nice if ctypes automagically turned the assignment into

   foofoo.bar = cast(babar,c_char_p)

There doesn't seem to be any other possible intent of this syntax. Color me stupid, but being new to ctypes, it took me approximately 30 hours to discover a method of such assignment that worked (babar.raw is not rejected, but not correct, debugging that was fun. I actually fell upon addressof(babar), which does also works, but is probably undesirable.)

It would be very friendly if ctypes would just do the right thing. It doesn't seem like there is any other possible intent of such a syntax.

[005e1818-9bcc-4f8b-82f5-f49496dd2c61]

The main reason for this issue is that internally ctypes doesn't consider c_char_p as a pointer type. This is a bit counter-intuitive, but c_char_p is treated as a simple type and has some other sugar built-in, like easier integration with Python strings. Alternative pointer type is POINTER(c_char), which allows assignment from array.

I would make this clear in the documentation. Even now, documentation says following about c_char_p:

Represents the C char * datatype when it points to a zero-terminated string. For a general character pointer that may also point to binary data, POINTER(c_char) must be used. The constructor accepts an integer address, or a string.

So there already is a hint that c_char_p has limited use for convenience. We should maybe make documentation more clear. If you want equivalent of C char* behavior, POINTER(c_char) is the way to go.

[meadori]

This is busted for plain old assignment too:

Python 3.3.0a0 (default:6374b4ffe00c, Sep  2 2011, 23:50:39) 
[GCC 4.6.0 20110603 (Red Hat 4.6.0-10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> buff = create_string_buffer(b'foo')
>>> p = c_char_p()
>>> p.value = addressof(buff)
>>> print(p.value)
b'foo'
>>> p.value = buff.value
>>> print(p.value)
b'foo'
>>> p.value = buff
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: string or integer address expected instead of c_char_Array_4 instance

I think having the conversion is an entirely reasonable request. It is the equivalent of:

    char buff[128] = "foo";
    char *p = buff;

in C. I imagine a lot of C programmers would expect this behavior.

Also, 'ctypes' already does this type of conversion for function parameters. Consider a function exported from a shared library 'libfoo' called 'print_string':

void print_string (char *str)
{
  printf ("%s\n", str);
}

The following all work fine:

>>> libfoo = CDLL("./libfoo.so.1.0")
>>> buff = create_string_buffer("foo")
>>> libfoo.print_string(buff)
foo
>>> libfoo.print_string("foo")
foo
>>> libfoo.print_string(addressof(buff))
foo

I am working on a patch for this. I will post it soon.

[005e1818-9bcc-4f8b-82f5-f49496dd2c61]

I believe there is a deeper issue here in ctypes design. Basically we provide both c_char_p and POINTER(c_char) which should behave exactly the same since both are the equivalent of char* in C but internally they have different implementations.

c_char_p is considered a "simple type" and I believe supports some conversions to and from Python strings while POINTER(c_char) is considered a pointer type which supports assignment from array etc.

I think a better fix would be to deprecate p_char_p or make it an equivalent of POINTER(c_char), otherwise we will have to do work on c_char_p to make it more like POINTER(c_char) when issues like this get opened and probably also make POINTER(c_char) more like c_char_p. Why not just have POINTER(c_char) which works as expected? I don't have all the historical context on why this pseudo-simple type was provided but I saw a couple of issues where people expect it to behave like a C char* but it won't because it is implemented as a convenience type with limited support.

[meadori]

Vlad, I agree that having both 'c_char_p' and 'POINTER(c_char)' will just be more work for two seemingly identical constructs. I don't fully understand why both would be needed either (or the implications of removing one of them or making them synonyms). I guess a little software archeology is in order.