MagicStack / asyncpg

A fast PostgreSQL Database Client Library for Python/asyncio.
Apache License 2.0
7.01k stars 402 forks source link

cockroachdb crdb - enums not working with prepared statements #658

Open toppk opened 4 years ago

toppk commented 4 years ago

enums have just been added to crdb. testing asyncpg shows an issue with prepared statements.

running this:

await conn.execute("INSERT INTO accounts2 (id, balance, mode) VALUES ($1, $2, $3)", 9, 2000, "active", ) throws

    statement = await self._protocol.prepare(stmt_name, query, timeout)
  File "asyncpg/protocol/protocol.pyx", line 163, in prepare
asyncpg.exceptions.FeatureNotSupportedError: CTEs may not be correlated

the prepared statement works fine in pgsql. enums work fine without prepared statements.

I have a silly bug exerciser, but needs some cleanup, you can see the relevant test2 functions if you need working code. i've tested psycopg2 and asyncpg against crdb and pgsql and out of the four runs, only asyncpg+crdb shows an issue.

crdbissues.py.txt

toppk commented 4 years ago

I've done some digging, it seems that the sql that is blowing up is the introspection that is being run. It seems that CTEs aren't fully implemented in crdb, for example:

https://github.com/cockroachdb/cockroach/issues/42540

It would seem that major restructuring of the introspection would be required. I will look for a quick workaround to punch through the data that is being sought.

toppk commented 4 years ago

Actually, in my case it's okay not to avoid using prepared parameter for the enum type, while still using prepared parameters for the rest of the arguments. i'm using prepared statements as a security method, not for any performance improvement, so this doesn't introduce any complexity on my end.

ale-dd commented 3 years ago

Hitting the same issue with introspection. I'm running asyncpg v0.22.0 against cockroach v20.2.4.

gnat commented 3 years ago

Can replicate this same issue with Cockroach DB v21.1.5

Enums work fine until passed into a prepared statement, which results in CTEs may not be correlated exception as described by @toppk

rafiss commented 3 years ago

Version 21.2 of CockroachDB will support correlated CTEs (https://github.com/cockroachdb/cockroach/pull/63956) -- hopefully that will help.

lacasaprivata2 commented 3 years ago

Same happens with

  SELECT DISTINCT ON (ACCOUNT.ID) * 
  FROM ACCOUNT
  WHERE ID = ANY($1 :: uuid[])
  ORDER BY ACCOUNT.ID, ACCOUNT.TIMESTAMP DESC
lacasaprivata2 commented 3 years ago

I think generally Cockroach isn't interoperable w/ asyncpg for non-typical queries (LEFT JOIN LATERAL, SELECT DISTINCT, etc...) from testing it on a project that require prepared staments

rafiss commented 3 years ago

Just to clarify, what is the error you get with the

  SELECT DISTINCT ON (ACCOUNT.ID) * 
  FROM ACCOUNT
  WHERE ID = ANY($1 :: uuid[])
  ORDER BY ACCOUNT.ID, ACCOUNT.TIMESTAMP DESC

query?