search

getredash / redash

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
http://redash.io/
BSD 2-Clause "Simplified" License
26.36k stars 4.37k forks source link

Recent ReHash db Migration throw an Error #7221

Open myonlylonely opened 4 hours ago

myonlylonely commented 4 hours ago

Issue Summary

The rehash db migration error. Related to https://github.com/getredash/redash/pull/7184 @arikfr

Steps to Reproduce

Recently I upgrade the docker image, and run an migration

docker-compose run --rm server manage db upgrade
[INFO][alembic.runtime.migration] Running upgrade 7205816877ec -> 9e8c841d1a30, fix_hash
...
Query 98 has parameters. Hash might be incorrect.
Updating hash for query 98 from 9841505301c4f5a5c5498aa0dfc7e0c6 to 9841505301c4f5a5c5498aa0dfc7e0c6
Traceback (most recent call last):
  File "/app/manage.py", line 9, in <module>
    manager()
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/flask/cli.py", line 357, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/flask_migrate/cli.py", line 134, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/usr/local/lib/python3.10/site-packages/flask_migrate/__init__.py", line 95, in wrapped
    f(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/flask_migrate/__init__.py", line 280, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/usr/local/lib/python3.10/site-packages/alembic/command.py", line 403, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.10/site-packages/alembic/script/base.py", line 583, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/app/migrations/env.py", line 93, in <module>
    run_migrations_online()
  File "/app/migrations/env.py", line 85, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.10/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.10/site-packages/alembic/runtime/migration.py", line 627, in run_migrations
    step.migration_fn(**kw)
  File "/app/migrations/versions/9e8c841d1a30_fix_hash.py", line 55, in upgrade
    new_hash = update_query_hash(record)
  File "/app/migrations/versions/9e8c841d1a30_fix_hash.py", line 26, in update_query_hash
    query_runner = get_query_runner(record['type'], {}) if record['type'] else BaseQueryRunner({})
  File "/app/redash/query_runner/__init__.py", line 433, in get_query_runner
    return query_runner_class(configuration)
  File "/app/redash/query_runner/elasticsearch.py", line 94, in __init__
    self.server_url = self.configuration["server"]
KeyError: 'server'

Technical details:

justinclift commented 3 hours ago

Oh that does look like it needs investigation. @arikfr You up for taking a look?

arikfr commented 3 hours ago

From a quick look it seems that some query runners (like the Elastic one) are not happy about empty options object…

I’ll need to think about how to address this. We can load the options but this might prove hard because it’s not a standard column. Need to check the code.

On Fri, Nov 15, 2024 at 7:26 AM myonlylonely @.***> wrote:

Issue Summary

The rehash db migration error. Related to #7184 https://github.com/getredash/redash/pull/7184 @arikfr https://github.com/arikfr Steps to Reproduce

Recently I upgrade the docker image, and run an migration

docker-compose run --rm server manage db upgrade

[INFO][alembic.runtime.migration] Running upgrade 7205816877ec -> 9e8c841d1a30, fix_hash ... Query 98 has parameters. Hash might be incorrect. Updating hash for query 98 from 9841505301c4f5a5c5498aa0dfc7e0c6 to 9841505301c4f5a5c5498aa0dfc7e0c6 Traceback (most recent call last): File "/app/manage.py", line 9, in manager() File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, *kwargs) File "/usr/local/lib/python3.10/site-packages/flask/cli.py", line 357, in decorator return __ctx.invoke(f, args, kwargs) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke return callback(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/flask_migrate/cli.py", line 134, in upgrade _upgrade(directory, revision, sql, tag, x_arg) File "/usr/local/lib/python3.10/site-packages/flask_migrate/init.py", line 95, in wrapped f(*args, kwargs) File "/usr/local/lib/python3.10/site-packages/flask_migrate/init.py", line 280, in upgrade command.upgrade(config, revision, sql=sql, tag=tag) File "/usr/local/lib/python3.10/site-packages/alembic/command.py", line 403, in upgrade script.run_env() File "/usr/local/lib/python3.10/site-packages/alembic/script/base.py", line 583, in run_env util.load_python_file(self.dir, "env.py") File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file module = load_module_py(module_id, path) File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py spec.loader.exec_module(module) # type: ignore File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/app/migrations/env.py", line 93, in run_migrations_online() File "/app/migrations/env.py", line 85, in run_migrations_online context.run_migrations() File "", line 8, in run_migrations File "/usr/local/lib/python3.10/site-packages/alembic/runtime/environment.py", line 948, in run_migrations self.get_context().run_migrations(kw) File "/usr/local/lib/python3.10/site-packages/alembic/runtime/migration.py", line 627, in run_migrations step.migration_fn(**kw) File "/app/migrations/versions/9e8c841d1a30_fix_hash.py", line 55, in upgrade new_hash = update_query_hash(record) File "/app/migrations/versions/9e8c841d1a30_fix_hash.py", line 26, in update_query_hash query_runner = get_query_runner(record['type'], {}) if record['type'] else BaseQueryRunner({}) File "/app/redash/query_runner/init.py", line 433, in get_query_runner return query_runner_class(configuration) File "/app/redash/query_runner/elasticsearch.py", line 94, in init__ self.server_url = self.configuration["server"] KeyError: 'server'

Technical details:

— Reply to this email directly, view it on GitHub https://github.com/getredash/redash/issues/7221, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAROLAZV7BGWBGA2RMRPA32AWAXBAVCNFSM6AAAAABR2NIVOWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGY3DANZXGI4DIMQ . You are receiving this because you were mentioned.Message ID: @.***>