Installation from PyPi, superset db upgrade fails

grundprinzip commented 6 months ago

Bug description

When trying to initialize the database, it fails

How to reproduce the bug

virtualenv -p python3.11 ~/Development/venv/superset
source ~/Development/venv/superset/bin/activate
pip install --upgrade 'apache-superset[databricks]'
Create the config file for the SQLLite path
superset db upgrade

Loaded your LOCAL configuration at [superset_config.py]
logging was configured successfully
2024-05-03 21:09:37,630:INFO:superset.utils.logging_configurator:logging was configured successfully
2024-05-03 21:09:37,634:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask_limiter/extension.py:337: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
  warnings.warn(
No PIL installation found
2024-05-03 21:09:37,717:INFO:superset.utils.screenshots:No PIL installation found
WARNI [alembic.env] SQLite Database support for metadata databases will         be removed in a future version of Superset.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 4e6a06bad7a8, Init
INFO  [alembic.runtime.migration] Running upgrade 4e6a06bad7a8 -> 5a7bad26f2a7, empty message
INFO  [alembic.runtime.migration] Running upgrade 5a7bad26f2a7 -> 1e2841a4128, empty message
INFO  [alembic.runtime.migration] Running upgrade 1e2841a4128 -> 2929af7925ed, TZ offsets in data sources
INFO  [alembic.runtime.migration] Running upgrade 2929af7925ed -> 289ce07647b, Add encrypted password field
INFO  [alembic.runtime.migration] Running upgrade 289ce07647b -> 1a48a5411020, adding slug to dash
INFO  [alembic.runtime.migration] Running upgrade 1a48a5411020 -> 315b3f4da9b0, adding log model
INFO  [alembic.runtime.migration] Running upgrade 315b3f4da9b0 -> 55179c7f25c7, sqla_descr
INFO  [alembic.runtime.migration] Running upgrade 55179c7f25c7 -> 12d55656cbca, is_featured
INFO  [alembic.runtime.migration] Running upgrade 12d55656cbca -> 2591d77e9831, user_id
Traceback (most recent call last):
  File "/Users/martin.grund/Development/venv/superset/bin/superset", line 8, in <module>
    sys.exit(superset())
             ^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask/cli.py", line 358, in decorator
    return __ctx.invoke(f, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask_migrate/cli.py", line 149, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask_migrate/__init__.py", line 98, in wrapped
    f(*args, **kwargs)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask_migrate/__init__.py", line 185, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/command.py", line 403, in upgrade
    script.run_env()
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/script/base.py", line 583, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/superset/extensions/../migrations/env.py", line 127, in <module>
    run_migrations_online()
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/superset/extensions/../migrations/env.py", line 119, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/runtime/migration.py", line 627, in run_migrations
    step.migration_fn(**kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/superset/migrations/versions/2015-12-15_17-02_2591d77e9831_user_id.py", line 34, in upgrade
    with op.batch_alter_table("tables") as batch_op:
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 144, in __exit__
    next(self.gen)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/operations/base.py", line 398, in batch_alter_table
    impl.flush()
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/operations/batch.py", line 142, in flush
    existing_table = Table(
                     ^^^^^^
  File "<string>", line 2, in __new__
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 618, in __new__
    with util.safe_reraise():
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 614, in __new__
    table._init(name, metadata, *args, **kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 689, in _init
    self._autoload(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 724, in _autoload
    conn_insp.reflect_table(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 795, in reflect_table
    self._reflect_fk(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 992, in _reflect_fk
    sa_schema.Table(
  File "<string>", line 2, in __new__
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 618, in __new__
    with util.safe_reraise():
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 614, in __new__
    table._init(name, metadata, *args, **kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 689, in _init
    self._autoload(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 724, in _autoload
    conn_insp.reflect_table(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 795, in reflect_table
    self._reflect_fk(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 992, in _reflect_fk
    sa_schema.Table(
  File "<string>", line 2, in __new__
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 618, in __new__
    with util.safe_reraise():
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 614, in __new__
    table._init(name, metadata, *args, **kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 689, in _init
    self._autoload(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 724, in _autoload
    conn_insp.reflect_table(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 789, in reflect_table
    raise exc.NoSuchTableError(table_name)
sqlalchemy.exc.NoSuchTableError: ab_user

Screenshots/recordings

No response

Superset version

4.0.0

Python version

3.11

Node version

16

Browser

Chrome

Additional context

No response

Checklist

[X] I have searched Superset docs and Slack and didn't find a solution to my problem.
[X] I have searched the GitHub issue tracker and didn't find a similar bug report.
[X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

hainenber commented 6 months ago

Can you initialize the DB without specifying the custom path for SQLite's db? May need you to delete whatever in ~/.superset to ensure a fresh initialization.

I suppose there's something that prevents ab_user from creating in the first place.

korhojoa commented 6 months ago

This happens without specifying a custom path. Just specifying the SECRET_KEY in mine causes superset db upgrade to fail with

ERROR [flask_migrate] Error: Multiple head revisions are present for given argument 'head'; please specify a specific target revision '<branchname>@head' to narrow to a specific head, or 'heads' for all heads

imancrsrk commented 6 months ago

I followed all your steps and just before running superset db upgrade, set the SECRET_KEY and it worked just fine for me.

grundprinzip commented 6 months ago

So, as it turns out, in my supersset_config with a SQLAlchemy path configured it will fail. If I comment out the following in my config file it works fine.


SECRET_KEY = 'BLA_BLA_BLA'
# Comment out this one to fail...
# SQLALCHEMY_DATABASE_URI = 'sqlite:///superset.db'
PREVENT_UNSAFE_DB_CONNECTIONS = False

imancrsrk commented 6 months ago

So, as it turns out, in my supersset_config with a SQLAlchemy path configured it will fail. If I comment out the following in my config file it works fine.
SECRET_KEY = 'BLA_BLA_BLA'
# Comment out this one to fail...
# SQLALCHEMY_DATABASE_URI = 'sqlite:///superset.db'
PREVENT_UNSAFE_DB_CONNECTIONS = False

So I've something like the below and it works just fine. I believe you are missing a couple of slashes -

SQLALCHEMY_DATABASE_URI = 'sqlite:////path-to-db/superset.db'

grundprinzip commented 6 months ago

This is my exact config file:

SECRET_KEY = 'BLA_BLA_BLA'
SQLALCHEMY_DATABASE_URI = 'sqlite:///superset.db'
PREVENT_UNSAFE_DB_CONNECTIONS = False

It works and throws the above error. it creates the superset db file so the path is correct but still throws.

grundprinzip commented 6 months ago

$ cat superset_config.py
SECRET_KEY = 'BLA_BLA_BLA'
SQLALCHEMY_DATABASE_URI = 'sqlite:///superset.db'
PREVENT_UNSAFE_DB_CONNECTIONS = False

$ superset db upgrade
Loaded your LOCAL configuration at [superset_config.py]
logging was configured successfully
2024-05-06 21:00:34,096:INFO:superset.utils.logging_configurator:logging was configured successfully
2024-05-06 21:00:34,098:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask_limiter/extension.py:337: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
  warnings.warn(
No PIL installation found
2024-05-06 21:00:34,168:INFO:superset.utils.screenshots:No PIL installation found
WARNI [alembic.env] SQLite Database support for metadata databases will         be removed in a future version of Superset.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 4e6a06bad7a8, Init
INFO  [alembic.runtime.migration] Running upgrade 4e6a06bad7a8 -> 5a7bad26f2a7, empty message
INFO  [alembic.runtime.migration] Running upgrade 5a7bad26f2a7 -> 1e2841a4128, empty message
INFO  [alembic.runtime.migration] Running upgrade 1e2841a4128 -> 2929af7925ed, TZ offsets in data sources
INFO  [alembic.runtime.migration] Running upgrade 2929af7925ed -> 289ce07647b, Add encrypted password field
INFO  [alembic.runtime.migration] Running upgrade 289ce07647b -> 1a48a5411020, adding slug to dash
INFO  [alembic.runtime.migration] Running upgrade 1a48a5411020 -> 315b3f4da9b0, adding log model
INFO  [alembic.runtime.migration] Running upgrade 315b3f4da9b0 -> 55179c7f25c7, sqla_descr
INFO  [alembic.runtime.migration] Running upgrade 55179c7f25c7 -> 12d55656cbca, is_featured
INFO  [alembic.runtime.migration] Running upgrade 12d55656cbca -> 2591d77e9831, user_id
Traceback (most recent call last):
  File "/Users/martin.grund/Development/venv/superset/bin/superset", line 8, in <module>
    sys.exit(superset())
             ^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask/cli.py", line 358, in decorator
    return __ctx.invoke(f, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask_migrate/cli.py", line 149, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask_migrate/__init__.py", line 98, in wrapped
    f(*args, **kwargs)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/flask_migrate/__init__.py", line 185, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/command.py", line 403, in upgrade
    script.run_env()
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/script/base.py", line 583, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/superset/extensions/../migrations/env.py", line 127, in <module>
    run_migrations_online()
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/superset/extensions/../migrations/env.py", line 119, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/runtime/migration.py", line 627, in run_migrations
    step.migration_fn(**kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/superset/migrations/versions/2015-12-15_17-02_2591d77e9831_user_id.py", line 34, in upgrade
    with op.batch_alter_table("tables") as batch_op:
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 144, in __exit__
    next(self.gen)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/operations/base.py", line 398, in batch_alter_table
    impl.flush()
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/alembic/operations/batch.py", line 142, in flush
    existing_table = Table(
                     ^^^^^^
  File "<string>", line 2, in __new__
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 618, in __new__
    with util.safe_reraise():
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 614, in __new__
    table._init(name, metadata, *args, **kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 689, in _init
    self._autoload(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 724, in _autoload
    conn_insp.reflect_table(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 795, in reflect_table
    self._reflect_fk(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 992, in _reflect_fk
    sa_schema.Table(
  File "<string>", line 2, in __new__
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 618, in __new__
    with util.safe_reraise():
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 614, in __new__
    table._init(name, metadata, *args, **kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 689, in _init
    self._autoload(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 724, in _autoload
    conn_insp.reflect_table(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 795, in reflect_table
    self._reflect_fk(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 992, in _reflect_fk
    sa_schema.Table(
  File "<string>", line 2, in __new__
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 618, in __new__
    with util.safe_reraise():
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 614, in __new__
    table._init(name, metadata, *args, **kw)
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 689, in _init
    self._autoload(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 724, in _autoload
    conn_insp.reflect_table(
  File "/Users/martin.grund/Development/venv/superset/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 789, in reflect_table
    raise exc.NoSuchTableError(table_name)
sqlalchemy.exc.NoSuchTableError: ab_user

sfirke commented 6 months ago

In case it helps in this case: I have a PR open to improve the docs for installing from PyPI. And if you have any feedback on those instructions, given that you are going through the process now, feel free to comment on that PR.

arabot777 commented 2 months ago

When you specify the relative path to sqlite:///superset.db, when you execute superset db upgrade, it will also initialize a superset.db file under pythonxx/site-packages/superset in venv environment. The ab_user table is in here. I think it has a bug, if you specify a relative path it will generate two superset.db files, and the table is also distributed in different files. Using absolute paths solves this problem: in custom superset_config.py

import os
# Gets the directory of the current script file
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
DB_PATH = os.path.abspath(os.path.join(SCRIPT_DIR, ".superset", "test.db"))
SQLALCHEMY_DATABASE_URI = f"sqlite:///{DB_PATH}?check_same_thread=false"
print(f"SQLite database path: {DB_PATH}")
PREVENT_UNSAFE_DB_CONNECTIONS = False

sorieux commented 2 months ago

When you specify the relative path to sqlite:///superset.db, when you execute superset db upgrade, it will also initialize a superset.db file under pythonxx/site-packages/superset in venv environment. The ab_user table is in here. I think it has a bug, if you specify a relative path it will generate two superset.db files, and the table is also distributed in different files. Using absolute paths solves this problem: in custom superset_config.py
import os
# Gets the directory of the current script file
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
DB_PATH = os.path.abspath(os.path.join(SCRIPT_DIR, ".superset", "test.db"))
SQLALCHEMY_DATABASE_URI = f"sqlite:///{DB_PATH}?check_same_thread=false"
print(f"SQLite database path: {DB_PATH}")
PREVENT_UNSAFE_DB_CONNECTIONS = False

Thank you, I had the same issue as Grundprinzip. Using the absolute path to the SQLite database resolved my problem.

rusackas commented 2 months ago

@sorieux @arabot777 do either of you see an opportunity to improve docs or file comments? We'd appreciate a PR or your input on the matter.

@grundprinzip can we effectively call this done if that solves the issue, or is there more to untangle here?

apache / superset