JohnOmernik / sqlalchemy-drill

Apache Drill Dialect for SQL Alchemy
Other
53 stars 35 forks source link

Error when reflecting a table #66

Closed amilamanoj closed 3 years ago

amilamanoj commented 3 years ago

Describe the bug Error when reflecting table when using Apache Drill

To Reproduce

from sqlalchemy import (
    create_engine,
    MetaData,
    Table
)
engine = create_engine('drill+sadrill://user:pass@host:port/dfs?use_ssl=True', echo = True)
mytable = Table(
        "sstest",
        MetaData(),
        schema="dfs.tmp",
        autoload_with=engine
    )

Error

2021-06-07 16:54:31,539 INFO sqlalchemy.engine.Engine SELECT SCHEMA_NAME, TYPE FROM INFORMATION_SCHEMA.`SCHEMATA` WHERE SCHEMA_NAME LIKE '%dfs.tmp%'
2021-06-07 16:54:31,540 INFO sqlalchemy.engine.Engine [raw sql] ()
2021-06-07 16:54:31,539 - SELECT SCHEMA_NAME, TYPE FROM INFORMATION_SCHEMA.`SCHEMATA` WHERE SCHEMA_NAME LIKE '%dfs.tmp%'
2021-06-07 16:54:31,540 - [raw sql] ()
2021-06-07 16:54:31,645 - SELECT `TABLE_NAME` FROM INFORMATION_SCHEMA.views WHERE table_schema='dfs.tmp'
2021-06-07 16:54:31,645 - [raw sql] ()
2021-06-07 16:54:31,645 INFO sqlalchemy.engine.Engine SELECT `TABLE_NAME` FROM INFORMATION_SCHEMA.views WHERE table_schema='dfs.tmp'
2021-06-07 16:54:31,645 INFO sqlalchemy.engine.Engine [raw sql] ()
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1770, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy_drill/drilldbapi/_drilldbapi.py", line 65, in func_wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy_drill/drilldbapi/_drilldbapi.py", line 165, in execute
    elif str(df[col_name].iloc[0]).startswith("[") and str(df[col_name].iloc[0]).endswith("]"):
  File "/usr/local/lib/python3.9/site-packages/pandas/core/indexing.py", line 895, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "/usr/local/lib/python3.9/site-packages/pandas/core/indexing.py", line 1501, in _getitem_axis
    self._validate_integer(key, axis)
  File "/usr/local/lib/python3.9/site-packages/pandas/core/indexing.py", line 1444, in _validate_integer
    raise IndexError("single positional indexer is out-of-bounds")
IndexError: single positional indexer is out-of-bounds

Versions.

Additional context Following query works and produces correct results:

engine.execute("select * from dfs.tmp.sstest")
amilamanoj commented 3 years ago

https://github.com/JohnOmernik/sqlalchemy-drill/pull/62 resolves this issue.