MagicStack / asyncpg

A fast PostgreSQL Database Client Library for Python/asyncio.
Apache License 2.0
6.99k stars 404 forks source link

`copy_from_query` does not handle `NoneType` parameters correctly #1179

Closed fobispotc closed 3 weeks ago

fobispotc commented 2 months ago

While using copy_from_query command, the handling of None values for the parameters is not processed correctly. The mogrify function in utils looks like it fails to convert the python NoneType into correponding NULL values.

One example could be:

SELECT $1::TEXT IS NULL OR $1 = 'foo'

This would return TRUE if $1 is set to NULL or if $1 is of value foo.. This works well when using fetchrow or similar commands, but fails with the copy_from _query command.

The error reported by Postgres is:


2024-09-08 14:42:51.918 PDT [2489056] srs@srs ERROR:  syntax error at or near "::" at character 19
2024-09-08 14:42:51.918 PDT [2489056] srs@srs STATEMENT:  COPY (
        SELECT ::TEXT IS NULL OR  = 'foo'
        ) TO STDOUT (FORMAT 'csv', HEADER True)

As you can see the parameter is replaced by ::TEXT and the second one to ` when it should've been:NULL` in BOTH cases.

fobispotc commented 2 months ago

The fix is simple, at the end of the _mogrify function in the asyncpg/utils directory, change the return to:

    return re.sub(
        r"\$(\d+)\b",
        lambda m: (
            textified[int(m.group(1)) - 1]
            if textified[int(m.group(1)) - 1] is not None
            else "NULL"
        ),
        query,
    )
fobispotc commented 2 months ago

PR added here: https://github.com/MagicStack/asyncpg/pull/1180