MagicStack / asyncpg

A fast PostgreSQL Database Client Library for Python/asyncio.
Apache License 2.0
6.99k stars 404 forks source link

Support inspecting return types of unnamed statements #982

Closed fvannee closed 1 year ago

fvannee commented 1 year ago

I'd like to be able to inspect the Postgres types of the returned columns when executing a query, without using a named prepared statement. The main reason for not using a named prepared statement is that it does not work with Pgbouncer in transaction mode, but I think in general it should be a useful feature to be able for people to see the types without having named prepared statements.

Internally, asyncpg has all this information. When you call connection.fetch() it calls_get_statement under the hood, which prepares an unnamed statement with all this information. Then it executes it, but only returns the final results, not the type information. So using fetch() is currently not possible to get this.

When using the prepare() call, it is possible to get this information. For example:

conn = await asyncpg.connect(
    statement_cache_size=0
)

async def prepare_and_fetch(conn: asyncpg.Connection, pg_query):
    stmt: asyncpg.prepared_stmt.PreparedStatement = await conn.prepare(pg_query, name=False)
    attrs = stmt.get_attributes()
    data = await stmt.fetch()

prepare_and_fetch(conn, 'select 1'):
prepare_and_fetch(conn, 'select 1'):

However, in this example, even though we set name=False, asyncpg will try to remove this unnamed statement on the second call to prepare_and_fetch. When stmt goes out of scope, it detaches from the state and calls self._connection._maybe_gc_stmt(self._state). The next time someone executes a query it'll try to free this unnamed function, but obviously there's no freeing to be done as Postgres automatically removed it already.

The minimal change to make this work is to just avoid freeing unnamed statements, which #981 implements. Obviously, there are still other things that (by design) don't work with unnamed prepared statements in combination with PgBouncer like this. For example, calling fetch() twice, might result in PgBouncer moving the second fetch() do a different connection. This is not more broken than it is now however.

    stmt: asyncpg.prepared_stmt.PreparedStatement = await conn.prepare(pg_query, name='abc')
    data = await stmt.fetch()
    data2 = await stmt.fetch() # this might run on a different connection thus failing

What do you think of #981 for a fix for unnamed statements?

fvannee commented 1 year ago

Thanks for merging! When is the next release scheduled? Or any chance a release could be made with this?