trinodb / trino-python-client

Python client for Trino
Apache License 2.0
307 stars 151 forks source link

SQLAlchemy Warning: index key was not located in columns for table #385

Closed ianmcook closed 11 months ago

ianmcook commented 1 year ago

Expected behavior

After I execute a CTAS statement to copy the table tpch.tiny.orders into a new table mycatalog.myschema.orders, I expect to be able to create a SQLAlchemy Table object on the new table with no SQLAlchemy warnings.

Actual behavior

SQLAlchemy raises these warnings when I create a Table object:

<ipython-input-1-be2d6890fdca>:21: SAWarning: index key 'record_count' was not located in columns for table 'orders'
  t = Table('orders', MetaData(schema='myschema'), autoload_with=conn)
<ipython-input-1-be2d6890fdca>:21: SAWarning: index key 'file_count' was not located in columns for table 'orders'
  t = Table('orders', MetaData(schema='myschema'), autoload_with=conn)
<ipython-input-1-be2d6890fdca>:21: SAWarning: index key 'total_size' was not located in columns for table 'orders'
  t = Table('orders', MetaData(schema='myschema'), autoload_with=conn)
<ipython-input-1-be2d6890fdca>:21: SAWarning: index key 'data' was not located in columns for table 'orders'
  t = Table('orders', MetaData(schema='myschema'), autoload_with=conn)

Steps To Reproduce

  1. Sign up for a free Starburst Galaxy account with email myusername@mydomain.com.
  2. Create a Starburst Galaxy cluster named mycluster.
  3. Create a new S3 catalog named mycatalog using defaults for all settings.
  4. Run the following example code:
    
    from urllib import parse
    from sqlalchemy import create_engine, text, Table, MetaData

engine = create_engine( 'trino://{user}:{password}@{host}:{port}/{catalog}'.format( user=parse.quote('myusername@mydomain.com/accountadmin', safe=''), password=parse.quote('mypassword', safe=''), host='mycluster.trino.galaxy.starburst.io', port=443, catalog='mycatalog' ), future=True )

with engine.connect() as conn: conn.execute(text('CREATE SCHEMA IF NOT EXISTS mycatalog.myschema')) conn.execute(text('CREATE TABLE IF NOT EXISTS mycatalog.myschema.orders AS SELECT * FROM tpch.tiny.orders')) conn.commit()

with engine.connect() as conn: t = Table('orders', MetaData(schema='myschema'), autoload_with=conn)


5. Observe the warnings raised by SQLAlchemy when the final line executes.

### Log output

_No response_

### Operating System

macOS Ventura 13.4

### Trino Python client version

0.326.0

### Trino Server version

417-galaxy-1-u132-gc56c0d7c25

### Python version

3.10.9

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!
ianmcook commented 1 year ago

cc @wolf-sigma

hovaesco commented 11 months ago

@ianmcook I cannot reproduce it, which SQLAlchemy version are you using? I was testing with 2.0.18.

ianmcook commented 11 months ago

@hovaesco I'm running Python 3.10.9 with SQLAlchemy 1.4.46.

hovaesco commented 11 months ago

@ianmcook here is the fix https://github.com/ibis-project/ibis/pull/6628

ianmcook commented 11 months ago

Thanks! I think that suppressing the warnings like in https://github.com/ibis-project/ibis/pull/6628 is a sufficient solution for us, so I'll close this.