crate / sqlalchemy-cratedb

SQLAlchemy dialect for CrateDB.
https://cratedb.com/docs/sqlalchemy-cratedb/
Apache License 2.0
3 stars 2 forks source link

SQLAlchemy create a new table failure #123

Closed Andyworker closed 6 years ago

Andyworker commented 6 years ago

Hey , Am i try create new table using with the engine.create_allreturns some errors :

Traceback (most recent call last):
  File "/root/workspaces/db_modals/modal.py", line 98, in <module>
    main()
  File "/root/workspaces/db_modals/modal.py", line 55, in main
    ModalBase.metadata.create_all(bind=engine)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 4004, in create_all
    tables=tables)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1940, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1549, in _run_visitor
    **kwargs).traverse_single(element)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
    _is_metadata_operation=True)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 791, in visit_table
    include_foreign_key_constraints=include_foreign_key_constraints
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1003, in _execute_ddl
    if not self.schema_for_object.is_default else None)
  File "<string>", line 1, in <lambda>
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 442, in compile
    return self._compiler(dialect, bind=bind, **kw)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 26, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 219, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 245, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 81, in _compiler_dispatch
    return meth(self, **kw)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 2447, in visit_create_table
    and not first_pk)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 245, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 81, in _compiler_dispatch
    return meth(self, **kw)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 2478, in visit_create_column
    first_pk=first_pk
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/crate/client/sqlalchemy/compiler.py", line 94, in get_column_specification
    self.dialect.type_compiler.process(column.type)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 293, in process
    return type_._compiler_dispatch(self, **kw)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 81, in _compiler_dispatch
    return meth(self, **kw)
  File "/root/workspaces/db_modals/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 2940, in visit_user_defined
    return type_.get_col_spec(**kw)
AttributeError: '_ObjectArray' object has no attribute 'get_col_spec'

how to fix it?

chaudum commented 6 years ago

Hi @Andyworker , thanks for reporting. Could you please also provide your table definition? From the stacktrace I assume it contains an ARRAY(OBJECT) column. Also, please post the versions of crate and sqlalchemy.

Andyworker commented 6 years ago

thx, You feed back !🙂

this is my env version:

SQLAlchemy==1.1.11 crate==0.21.2

this is my modal defined:

from sqlalchemy import create_engine
from crate.client.sqlalchemy.types import Object, ObjectArray
from sqlalchemy import Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.pool import QueuePool
from sqlalchemy.types import *
from sqlalchemy.orm import Session
from datetime import datetime
engine = create_engine('crate://', connect_args={
    'servers': ['localhost:4200']
},  poolclass=QueuePool, pool_size=1024)
ModalBase = declarative_base()
crate_session = Session(bind=engine)

def gen_uuid():
    return str(uuid.uuid4())

class TestDB(ModalBase):

    __tablename__ = 'testdb'

    uid = Column(String, primary_key=True, default=gen_uuid(), unique=True)
    task_uuid = Column(String, nullable=False, unique=True)
    datatype_version = Column(FLOAT, default=1.0)
    datasource = Column(String)
    host = Column(String, nullable=False)
    port = Column(Integer, nullable=False)
    country = Column(String)
    province = Column(String)
    city = Column(String)
    ISP = Column(String)
    latitude = Column(FLOAT)
    longitude = Column(FLOAT)
    service_name = Column(String)
    service_version = Column(String)
    service_protocol = Column(String)
    # tcp: 1, udp: 0
    protocol_type = Column(Integer, default=1)
    is_web_service = Column(Boolean, default=False)
    system_info = Column(Object)
    web_mix_info = Column(Object)
    ssl_mix_info = Column(Object)
    response = Column(String)
    tags = Column(ObjectArray)
    last_modified = Column(TIMESTAMP, default=datetime.now())

def main():
    db_owner = TestDB()
    ModalBase.metadata.create_all(bind=engine)

if __name__ == '__main__':
    main()

My CrateDB veriosn: 2.3.3

mfussenegger commented 6 years ago

Thanks for the schema definition. We're able to reproduce the issue and will take a look at it.

mfussenegger commented 6 years ago

We've a fix for this (https://github.com/crate/crate-python/pull/277) which will be released with the next hotfix release.

Andyworker commented 6 years ago

Nice, thx your job