Could not convert Python object into cstring

[xerial]

Do you have any idea why this error happens?

2015-04-15_01:10:20.34347 LOG:  statement: select * from pg_temp.presto_fetch()
2015-04-15_01:10:44.97052 ERROR:  could not convert Python object into cstring: Python string representation appears to contain null bytes
2015-04-15_01:10:44.97055 CONTEXT:  while creating return value
2015-04-15_01:10:44.97055       PL/Python function "presto_fetch"

[frsyuki]

The error says that a string object includes null (\\0) character. Does the query result include such string object?

[xerial]

I'll check. Thanks!

[frsyuki]

If yes, you need some normalization code around here:

• https://github.com/treasure-data/prestogres/blob/master/prestogres/pgsql/prestogres.py#L128
• https://github.com/treasure-data/prestogres/blob/master/prestogres/pgsql/prestogres.py#L136

[xerial]

Yes. The query results contain several null characters:

presto> SELECT td_client_id FROM (some table) where regexp_like(td_client_id, '\00');
                  td_client_id
-------------------------------------------------
 xxxxxxxxxxxxxxx7e3cebf9fe^@^@^@^@
 xxxxxxxxxxxxxxx1a-871b-3d648572fee2^@
 xxxxxxxxxxxxxxx230-a318-31be55551193^@^@
 ...

[frsyuki]

I see. It means that Presto can contain \0 character in a string but PostgreSQL can't. Prestogres needs to manage the difference.

[frsyuki]

Could you paste detailed stacktrace?

[xerial]

I checked the log, but couldn't find any stacktrace around the time error occurred. Our td-presto may also need to handle trailing null values.

[frsyuki]

ok, try this: https://github.com/treasure-data/prestogres/pull/31