nackjicholson / aiosql

Simple SQL in Python
https://nackjicholson.github.io/aiosql
Other
1.29k stars 58 forks source link

[DuckDB] aiosql stops working with DuckDB >= 1.1.0 #203

Open maver1ck opened 3 hours ago

maver1ck commented 3 hours ago

Hi, I have following problem. I'm using aiosql to manage queries on top of DuckDB. With DuckDB upgrade 1.0.0 -> 1.1.0 this solution stops working.

Sample code:

import duckdb
import aiosql
import pandas as pd

print(duckdb.__version__)
print(aiosql.__version__)
print(pd.__version__)

data = {
    'ID': [1, 2, 3, 4, 5],
}

# Creating the DataFrame
df = pd.DataFrame(data)

queries = aiosql.from_path('test.sql', "duckdb")

with duckdb.connect(":memory:") as conn:
    print(conn.sql("SELECT COUNT(*) FROM df")) 
    print("aiosql result:", list(queries.test_sql(conn)))

test.sql

-- name: test_sql
SELECT COUNT(*) FROM df;

Output DuckDB version 1.0.0:

1.0.0
12.0
2.2.2
┌──────────────┐
│ count_star() │
│    int64     │
├──────────────┤
│            5 │
└──────────────┘

aiosql result: [(5,)]

Output DuckDB version 1.1.1:

1.1.1
12.0
2.2.2
┌──────────────┐
│ count_star() │
│    int64     │
├──────────────┤
│            5 │
└──────────────┘

---------------------------------------------------------------------------
CatalogException                          Traceback (most recent call last)
Cell In[1], line 18
     14 df = pd.DataFrame(data)
     16 queries = aiosql.from_path('test.sql', "duckdb")
---> 18 with duckdb.connect(":memory:") as conn:
     19     print(conn.sql("SELECT COUNT(*) FROM df")) 
     20     print("aiosql result:", list(queries.test_sql(conn)))

Cell In[1], line 20
     18 with duckdb.connect(":memory:") as conn:
     19     print(conn.sql("SELECT COUNT(*) FROM df")) 
---> 20     print("aiosql result:", list(queries.test_sql(conn)))

File [~/.virtualenvs/data-cleansing-kvgw/lib/python3.12/site-packages/aiosql/adapters/duckdb.py:44](http://localhost:8888/~/.virtualenvs/data-cleansing-kvgw/lib/python3.12/site-packages/aiosql/adapters/duckdb.py#line=43), in DuckDBAdapter.select(self, conn, query_name, sql, parameters, record_class)
     42 cur = self._cursor(conn)
     43 try:
---> 44     cur.execute(sql, parameters)
     45     if record_class is None:
     46         first = True

CatalogException: Catalog Error: Table with name df does not exist!
Did you mean "pg_depend"?
LINE 1: SELECT COUNT(*) FROM df;

This is probably related to following issue in DuckDB: https://github.com/duckdb/duckdb/issues/13836 It also doesn't work with conn.execute("set python_scan_all_frames=true")

maver1ck commented 2 hours ago

I created PR to fix this issue. https://github.com/nackjicholson/aiosql/issues/203