chdb-io / chdb

chDB is an in-process OLAP SQL Engine 🚀 powered by ClickHouse
https://clickhouse.com/docs/en/chdb
Apache License 2.0
2.03k stars 72 forks source link

DBAPI doesn't support CREATE TABLE or other none-returning queries #162

Closed yunyu closed 8 months ago

yunyu commented 9 months ago

Describe what's wrong

CHDB's dbapi implementation does not work for basic queries like CREATE TABLE that return no data. I am using DBAPI for sqlalchemy compatibility.

Does it reproduce on recent release?

Yes, reproduced on chdb==1.0.2

How to reproduce

import chdb.dbapi as dbapi
conn1 = dbapi.connect()
cur1 = conn1.cursor()
cur1.execute("""CREATE TABLE rate (
    day Date, 
    value Int32
) ENGINE = Memory""")

Expected behavior

Execute succeeds without an exception

Error message and/or stacktrace

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File ~/venv/lib/python3.11/site-packages/chdb/dbapi/connections.py:185, in CHDBResult.read(self)
    184 try:
--> 185     data = json.loads(self.connection.resp)
    186 except Exception as error:

File /usr/local/lib/python3.11/json/__init__.py:339, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    338 if not isinstance(s, (bytes, bytearray)):
--> 339     raise TypeError(f'the JSON object must be str, bytes or bytearray, '
    340                     f'not {s.__class__.__name__}')
    341 s = s.decode(detect_encoding(s), 'surrogatepass')

TypeError: the JSON object must be str, bytes or bytearray, not NoneType

During handling of the above exception, another exception occurred:
...
File ~/venv/lib/python3.11/site-packages/chdb/dbapi/connections.py:187, in CHDBResult.read(self)
    185     data = json.loads(self.connection.resp)
    186 except Exception as error:
--> 187     raise err.InterfaceError("Unsupported response format:" % error)
    189 try:
    190     self.field_count = len(data["meta"])

TypeError: not all arguments converted during string formatting

Additional context

This seems to work fine in the non DBAPI implementation.

yunyu commented 9 months ago

It looks like this behavior is caused by:

https://github.com/chdb-io/chdb/blob/b76b240cc45c96d796c2a4783aa8856a6099c9d0/chdb/dbapi/connections.py#L123

Which calls _read_query_result() even before fetchone/etc are called. I'm not sure if this behavior is correct, but in this case self.connection.resp is none presumably because CREATE TABLE doesn't return anything.