search

amancevice / docker-superset

Docker image for Airbnb's Superset
https://hub.docker.com/r/amancevice/superset/
MIT License
957 stars 416 forks source link

[BUG] Async query with celery doesn't work in SQL Lab #132

Open wegamekinglc opened 5 years ago

wegamekinglc commented 5 years ago

my docker compose settings:

version: '3'
services:
  redis:
    image: redis
    restart: always
    volumes:
      - redis:/data
  postgres:
    image: postgres
    restart: always
    environment:
      POSTGRES_DB: superset
      POSTGRES_PASSWORD: superset
      POSTGRES_USER: superset
    volumes:
      - ./postgres:/var/lib/postgresql/data
  superset:
    image: amancevice/superset
    restart: always
    depends_on:
      - postgres
      - redis
    environment:
      MAPBOX_API_KEY: ${MAPBOX_API_KEY}
    ports:
      - "8088:8088"
    volumes:
      - ./superset:/etc/superset
  worker:
    image: amancevice/superset
    restart: always
    depends_on:
      - postgres
      - redis
    environment:
      MAPBOX_API_KEY: ${MAPBOX_API_KEY}
    volumes:
      - ./superset:/etc/superset
    command: celery worker
volumes:
  postgres:
  redis:

And the error message is:

<html> <body> <h1>Sorry, something went wrong</h1> <h3>500 - Internal Server Error</h3> <hr> <h2>Stacktrace</h2> <hr> <code> <pre> Traceback (most recent call last): File &#34;/usr/local/lib/python3.6/site-packages/kombu/utils/functional.py&#34;, line 42, in __call__ return self.__value__ AttributeError: &#39;ChannelPromise&#39; object has no attribute &#39;__value__&#39; During handling of the above exception, another exception occurred: Traceback (most recent call last): File &#34;/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/base.py&#34;, line 921, in create_channel return self._avail_channels.pop() IndexError: pop from empty list During handling of the above exception, another exception occurred: Traceback (most recent call last): File &#34;/usr/local/lib/python3.6/site-packages/superset/views/core.py&#34;, line 2589, in sql_json start_time=now_as_float(), File &#34;/usr/local/lib/python3.6/site-packages/celery/app/task.py&#34;, line 427, in delay return self.apply_async(args, kwargs) File &#34;/usr/local/lib/python3.6/site-packages/celery/app/task.py&#34;, line 570, in apply_async **options File &#34;/usr/local/lib/python3.6/site-packages/celery/app/base.py&#34;, line 756, in send_task amqp.send_task_message(P, name, message, **options) File &#34;/usr/local/lib/python3.6/site-packages/celery/app/amqp.py&#34;, line 552, in send_task_message **properties File &#34;/usr/local/lib/python3.6/site-packages/kombu/messaging.py&#34;, line 181, in publish exchange_name, declare, File &#34;/usr/local/lib/python3.6/site-packages/kombu/connection.py&#34;, line 510, in _ensured return fun(*args, **kwargs) File &#34;/usr/local/lib/python3.6/site-packages/kombu/messaging.py&#34;, line 187, in _publish channel = self.channel File &#34;/usr/local/lib/python3.6/site-packages/kombu/messaging.py&#34;, line 209, in _get_channel channel = self._channel = channel() File &#34;/usr/local/lib/python3.6/site-packages/kombu/utils/functional.py&#34;, line 44, in __call__ value = self.__value__ = self.__contract__() File &#34;/usr/local/lib/python3.6/site-packages/kombu/messaging.py&#34;, line 224, in &lt;lambda&gt; channel = ChannelPromise(lambda: connection.default_channel) File &#34;/usr/local/lib/python3.6/site-packages/kombu/connection.py&#34;, line 852, in default_channel self.ensure_connection(**conn_opts) File &#34;/usr/local/lib/python3.6/site-packages/kombu/connection.py&#34;, line 422, in ensure_connection callback, timeout=timeout) File &#34;/usr/local/lib/python3.6/site-packages/kombu/utils/functional.py&#34;, line 343, in retry_over_time return fun(*args, **kwargs) File &#34;/usr/local/lib/python3.6/site-packages/kombu/connection.py&#34;, line 275, in connect return self.connection File &#34;/usr/local/lib/python3.6/site-packages/kombu/connection.py&#34;, line 823, in connection self._connection = self._establish_connection() File &#34;/usr/local/lib/python3.6/site-packages/kombu/connection.py&#34;, line 778, in _establish_connection conn = self.transport.establish_connection() File &#34;/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/base.py&#34;, line 941, in establish_connection self._avail_channels.append(self.create_channel(self)) File &#34;/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/base.py&#34;, line 923, in create_channel channel = self.Channel(connection) File &#34;/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py&#34;, line 496, in __init__ self.Client = self._get_client() File &#34;/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py&#34;, line 957, in _get_client &#39;You have {0.__version__}&#39;.format(redis)) kombu.exceptions.VersionMismatch: Redis transport requires redis-py versions 3.2.0 or later. You have 2.10.5 During handling of the above exception, another exception occurred: Traceback (most recent call last): File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py&#34;, line 1244, in _execute_context cursor, statement, parameters, context File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py&#34;, line 552, in do_execute cursor.execute(statement, parameters) psycopg2.ProgrammingError: can&#39;t adapt type &#39;LazyString&#39; The above exception was the direct cause of the following exception: Traceback (most recent call last): File &#34;/usr/local/lib/python3.6/site-packages/flask/app.py&#34;, line 2446, in wsgi_app response = self.full_dispatch_request() File &#34;/usr/local/lib/python3.6/site-packages/flask/app.py&#34;, line 1951, in full_dispatch_request rv = self.handle_user_exception(e) File &#34;/usr/local/lib/python3.6/site-packages/flask/app.py&#34;, line 1820, in handle_user_exception reraise(exc_type, exc_value, tb) File &#34;/usr/local/lib/python3.6/site-packages/flask/_compat.py&#34;, line 39, in reraise raise value File &#34;/usr/local/lib/python3.6/site-packages/flask/app.py&#34;, line 1949, in full_dispatch_request rv = self.dispatch_request() File &#34;/usr/local/lib/python3.6/site-packages/flask/app.py&#34;, line 1935, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File &#34;/usr/local/lib/python3.6/site-packages/flask_appbuilder/security/decorators.py&#34;, line 168, in wraps return f(self, *args, **kwargs) File &#34;/usr/local/lib/python3.6/site-packages/superset/utils/log.py&#34;, line 60, in wrapper value = f(*args, **kwargs) File &#34;/usr/local/lib/python3.6/site-packages/superset/views/core.py&#34;, line 2600, in sql_json session.commit() File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py&#34;, line 1027, in commit self.transaction.commit() File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py&#34;, line 494, in commit self._prepare_impl() File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py&#34;, line 473, in _prepare_impl self.session.flush() File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py&#34;, line 2459, in flush self._flush(objects) File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py&#34;, line 2597, in _flush transaction.rollback(_capture_exception=True) File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py&#34;, line 68, in __exit__ compat.reraise(exc_type, exc_value, exc_tb) File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py&#34;, line 153, in reraise raise value File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py&#34;, line 2557, in _flush flush_context.execute() File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py&#34;, line 422, in execute rec.execute(self) File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py&#34;, line 589, in execute uow, File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py&#34;, line 236, in save_obj update, File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py&#34;, line 996, in _emit_update_statements statement, multiparams File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py&#34;, line 988, in execute return meth(self, multiparams, params) File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py&#34;, line 287, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py&#34;, line 1107, in _execute_clauseelement distilled_params, File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py&#34;, line 1248, in _execute_context e, statement, parameters, cursor, context File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py&#34;, line 1466, in _handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py&#34;, line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py&#34;, line 152, in reraise raise value.with_traceback(tb) File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py&#34;, line 1244, in _execute_context cursor, statement, parameters, context File &#34;/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py&#34;, line 552, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can&#39;t adapt type &#39;LazyString&#39; [SQL: UPDATE query SET status=%(status)s, &#34;limit&#34;=%(limit)s, error_message=%(error_message)s, changed_on=%(changed_on)s WHERE query.id = %(query_id)s] [parameters: {&#39;status&#39;: &#39;failed&#39;, &#39;limit&#39;: 1000, &#39;error_message&#39;: l&#39;Failed to start remote query on a worker. Tell your administrator to verify the availability of the message queue.&#39;, &#39;changed_on&#39;: datetime.datetime(2019, 8, 23, 9, 26, 42, 948735), &#39;query_id&#39;: 30}] (Background on this error at: http://sqlalche.me/e/f405) </pre> </code> </body> </html>
caiobelfort commented 5 years ago

Maybe this is a mismatch between kombu and redis python packages. https://github.com/celery/celery/issues/5369

wegamekinglc commented 5 years ago

Maybe this is a mismatch between kombu and redis python packages. https://github.com/celery/celery/issues/5369

Thank you very much @caiobelfort. So should I upgrade the redis py version in Dockefile and rebuild the image for superset to solve the problem?

prideloki commented 5 years ago

@wegamekinglc hi, got the same issue. need to update redis version