chtd / psycopg2cffi

Port to cffi with some speed improvements
Other
177 stars 43 forks source link

Array types are sometimes returned as lists, sometimes as strings #93

Open wiml opened 6 years ago

wiml commented 6 years ago

If I perform a query that returns an array (possibly only if it's of a datatype that doesn't have a native python conversion and so is represented as a string), then empty arrays are (correctly) returned as empty lists, but nonempty arrays are (incorrectly) returned as a single string in postgres array syntax.

What I think should happen is that any column of an ARRAY type should always return a list (or None for NULL), and the list elements should be whatever they would be if they were returned as individual objects.

For example, here's what happens if I select from a simple "hosts" table:

>>> cu.execute("select address from hosts") ; cu.fetchall()
[([],), ('{10.1.0.1}',), ('{192.168.2.1,192.168.2.2}',)]

Notice that the value is a python list for the empty array row, but is a python string for the non-empty array rows.

In contrast, the psycopg2 module returns the values I expect:

>>> cu.execute("select address from hosts") ; cu.fetchall()
[([],), (['10.1.0.1'],), (['192.168.2.1', '192.168.2.2'],)]

(On the other hand if I have a column of type TEXT ARRAY or INTEGER ARRAY, then I get the behavior I expect.)

The database table was created as:

create table "hosts" ( "name" text, "address" inet array not null );
insert into hosts ( name, address ) values ('foo', '{ }'), ('bar', '{ 10.1.0.1 }'), ('baz', '{192.168.2.1, 192.168.2.2}');

Versions: psycopg2cffi 2.7.7, postgres 10.1, psycopg2 2.7.3.2, python 3.6.3

wiml commented 6 years ago

Another detail: registering a converter hides the problem:

>>> psycopg2cffi.extras.register_inet(cfu)
<psycopg2cffi._impl.typecasts.Type object at 0x10899f1d0>
>>> cfu.execute("select address from hosts") ; cfu.fetchall()
[([],), ([Inet('10.1.0.1')],), ([Inet('192.168.2.1'), Inet('192.168.2.2')],)]

Another workaround is to select CAST(address AS text array) to get the list-of-strings behavior.