Closed Galeos93 closed 1 year ago
Hi @Galeos93! Sorry you ran into this issue. From what I can tell from the stack trace it appears that one of our internal database migrations failed midway at some point. If you're not too deep into the weeds of your playing round with ZenML -- and it appears you're at the beginning of your journey since you're running the quickstart -- I'd suggest that the quickest way to fix this is just to get a fresh start with a new database. If you run zenml clean
that should take care of that. If you need to do it manually I can explain that too, but let me know if you run into any further issues or require help with doing it manually (instead of through the ZenML CLI).
Hi @strickvl, thanks for the help. Unfortunately, I have tried cleaning everything out and I still get the same error. These are the steps I have followed:
.zen
folders of the repository where I am workingrm -rf /home/agarcia/.config/zenml
python -I -m pip install pytest -r requirements.txt
zenml integration install sklearn mlflow evidently -y
zenml init
Maybe I am missing some key step? Can you reproduce this?
I have also tried the above steps but using zenml clean
before the last two steps. I get a different error when I do zenml clean
:
agarcia@agarcia-MS-7A34:~/repos/zenml_demo$ zenml clean
The local ZenML dashboard is not running.
DANGER: This will completely delete all artifacts, metadata and stacks
ever created during the use of ZenML. Pipelines and stack components running non-
locally will still exist. Please delete those manually.
Are you sure you want to proceed? [y/n]: y
Deleting 'local_stores' directory from global config.
Deleted global ZenML config from /home/agarcia/.config/zenml.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1900 in │
│ _execute_context │
│ │
│ 1897 │ │ │ │ │ │ │ evt_handled = True │
│ 1898 │ │ │ │ │ │ │ break │
│ 1899 │ │ │ │ if not evt_handled: │
│ ❱ 1900 │ │ │ │ │ self.dialect.do_execute( │
│ 1901 │ │ │ │ │ │ cursor, statement, parameters, context │
│ 1902 │ │ │ │ │ ) │
│ 1903 │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py:736 in do_execute │
│ │
│ 733 │ │ cursor.executemany(statement, parameters) │
│ 734 │ │
│ 735 │ def do_execute(self, cursor, statement, parameters, context=None): │
│ ❱ 736 │ │ cursor.execute(statement, parameters) │
│ 737 │ │
│ 738 │ def do_execute_no_params(self, cursor, statement, context=None): │
│ 739 │ │ cursor.execute(statement) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OperationalError: no such column: True
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/base_zen_store.py:143 in │
│ __init__ │
│ │
│ 140 │ │ super().__init__(**kwargs) │
│ 141 │ │ │
│ 142 │ │ try: │
│ ❱ 143 │ │ │ self._initialize() │
│ 144 │ │ except Exception as e: │
│ 145 │ │ │ raise RuntimeError( │
│ 146 │ │ │ │ f"Error initializing {self.type.value} store with URL " │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/sql_zen_store.py:840 in │
│ _initialize │
│ │
│ 837 │ │ │ not self.skip_migrations │
│ 838 │ │ │ and ENV_ZENML_DISABLE_DATABASE_MIGRATION not in os.environ │
│ 839 │ │ ): │
│ ❱ 840 │ │ │ self.migrate_database() │
│ 841 │ │
│ 842 │ def _create_mysql_database( │
│ 843 │ │ self, │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/sql_zen_store.py:908 in │
│ migrate_database │
│ │
│ 905 │ │ │ if self.alembic.db_is_empty(): │
│ 906 │ │ │ │ # Case 1: the database is empty. We can just create the │
│ 907 │ │ │ │ # tables from scratch with alembic. │
│ ❱ 908 │ │ │ │ self.alembic.upgrade() │
│ 909 │ │ │ else: │
│ 910 │ │ │ │ # Case 2: the database is not empty, but has never been │
│ 911 │ │ │ │ # migrated with alembic before. We need to create the alembic │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/migrations/alembic.py:201 in │
│ upgrade │
│ │
│ 198 │ │ │ │ revision, rev # type:ignore [arg-type] │
│ 199 │ │ │ ) │
│ 200 │ │ │
│ ❱ 201 │ │ self.run_migrations(do_upgrade) │
│ 202 │ │
│ 203 │ def downgrade(self, revision: str) -> None: │
│ 204 │ │ """Revert the database to a previous version. │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/migrations/alembic.py:152 in │
│ run_migrations │
│ │
│ 149 │ │ │ ) │
│ 150 │ │ │ │
│ 151 │ │ │ with self.environment_context.begin_transaction(): │
│ ❱ 152 │ │ │ │ self.environment_context.run_migrations() │
│ 153 │ │
│ 154 │ def current_revisions(self) -> List[str]: │
│ 155 │ │ """Get the current database revisions. │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/runtime/environment.py:853 in │
│ run_migrations │
│ │
│ 850 │ │ """ │
│ 851 │ │ assert self._migration_context is not None │
│ 852 │ │ with Operations.context(self._migration_context): │
│ ❱ 853 │ │ │ self.get_context().run_migrations(**kw) │
│ 854 │ │
│ 855 │ def execute(self, sql, execution_options=None): │
│ 856 │ │ """Execute the given SQL using the current change context. │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/runtime/migration.py:623 in │
│ run_migrations │
│ │
│ 620 │ │ │ │ │ self.impl.static_output( │
│ 621 │ │ │ │ │ │ "-- Running %s" % (step.short_log,) │
│ 622 │ │ │ │ │ ) │
│ ❱ 623 │ │ │ │ step.migration_fn(**kw) │
│ 624 │ │ │ │ │
│ 625 │ │ │ │ # previously, we wouldn't stamp per migration │
│ 626 │ │ │ │ # if we were in a transaction, however given the more │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/migrations/versions/d26471b6fe │
│ 8f_update_build_filtering.py:45 in upgrade │
│ │
│ 42 │ │ │ │ "checksum", sqlmodel.sql.sqltypes.AutoString(), nullable=True │
│ 43 │ │ │ ) │
│ 44 │ │ ) │
│ ❱ 45 │ op.execute("UPDATE pipeline_build SET contains_code = True") │
│ 46 │ with op.batch_alter_table("pipeline_build", schema=None) as batch_op: │
│ 47 │ │ batch_op.alter_column( │
│ 48 │ │ │ "contains_code", │
│ <string>:8 in execute │
│ <string>:3 in execute │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/operations/ops.py:2414 in execute │
│ │
│ 2411 │ │ :meth:`sqlalchemy.engine.Connection.execution_options`. │
│ 2412 │ │ """ │
│ 2413 │ │ op = cls(sqltext, execution_options=execution_options) │
│ ❱ 2414 │ │ return operations.invoke(op) │
│ 2415 │
│ 2416 │
│ 2417 class OpContainer(MigrateOperation): │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/operations/base.py:399 in invoke │
│ │
│ 396 │ │ fn = self._to_impl.dispatch( │
│ 397 │ │ │ operation, self.migration_context.impl.__dialect__ │
│ 398 │ │ ) │
│ ❱ 399 │ │ return fn(self, operation) │
│ 400 │ │
│ 401 │ def f(self, name: str) -> "conv": │
│ 402 │ │ """Indicate a string name that has already had a naming convention │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/operations/toimpl.py:207 in execute_sql │
│ │
│ 204 def execute_sql( │
│ 205 │ operations: "Operations", operation: "ops.ExecuteSQLOp" │
│ 206 ) -> None: │
│ ❱ 207 │ operations.migration_context.impl.execute( │
│ 208 │ │ operation.sqltext, execution_options=operation.execution_options │
│ 209 │ ) │
│ 210 │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/ddl/impl.py:202 in execute │
│ │
│ 199 │ │ sql: Union["Update", "TextClause", str], │
│ 200 │ │ execution_options: None = None, │
│ 201 │ ) -> None: │
│ ❱ 202 │ │ self._exec(sql, execution_options) │
│ 203 │ │
│ 204 │ def alter_column( │
│ 205 │ │ self, │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/ddl/impl.py:195 in _exec │
│ │
│ 192 │ │ │ │ assert isinstance(multiparams, tuple) │
│ 193 │ │ │ │ multiparams += (params,) │
│ 194 │ │ │ │
│ ❱ 195 │ │ │ return conn.execute(construct, multiparams) │
│ 196 │ │
│ 197 │ def execute( │
│ 198 │ │ self, │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/future/engine.py:280 in execute │
│ │
│ 277 │ │ :return: a :class:`_engine.Result` object. │
│ 278 │ │ │
│ 279 │ │ """ │
│ ❱ 280 │ │ return self._execute_20( │
│ 281 │ │ │ statement, parameters, execution_options or NO_OPTIONS │
│ 282 │ │ ) │
│ 283 │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1705 in _execute_20 │
│ │
│ 1702 │ │ │ │ exc.ObjectNotExecutableError(statement), replace_context=err │
│ 1703 │ │ │ ) │
│ 1704 │ │ else: │
│ ❱ 1705 │ │ │ return meth(self, args_10style, kwargs_10style, execution_options) │
│ 1706 │ │
│ 1707 │ def exec_driver_sql( │
│ 1708 │ │ self, statement, parameters=None, execution_options=None │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py:333 in │
│ _execute_on_connection │
│ │
│ 330 │ │ self, connection, multiparams, params, execution_options, _force=False │
│ 331 │ ): │
│ 332 │ │ if _force or self.supports_execution: │
│ ❱ 333 │ │ │ return connection._execute_clauseelement( │
│ 334 │ │ │ │ self, multiparams, params, execution_options │
│ 335 │ │ │ ) │
│ 336 │ │ else: │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1572 in │
│ _execute_clauseelement │
│ │
│ 1569 │ │ │ schema_translate_map=schema_translate_map, │
│ 1570 │ │ │ linting=self.dialect.compiler_linting | compiler.WARN_LINTING, │
│ 1571 │ │ ) │
│ ❱ 1572 │ │ ret = self._execute_context( │
│ 1573 │ │ │ dialect, │
│ 1574 │ │ │ dialect.execution_ctx_cls._init_compiled, │
│ 1575 │ │ │ compiled_sql, │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1943 in │
│ _execute_context │
│ │
│ 1940 │ │ │ │ │ branched.close() │
│ 1941 │ │ │
│ 1942 │ │ except BaseException as e: │
│ ❱ 1943 │ │ │ self._handle_dbapi_exception( │
│ 1944 │ │ │ │ e, statement, parameters, cursor, context │
│ 1945 │ │ │ ) │
│ 1946 │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:2124 in │
│ _handle_dbapi_exception │
│ │
│ 2121 │ │ │ if newraise: │
│ 2122 │ │ │ │ util.raise_(newraise, with_traceback=exc_info[2], from_=e) │
│ 2123 │ │ │ elif should_wrap: │
│ ❱ 2124 │ │ │ │ util.raise_( │
│ 2125 │ │ │ │ │ sqlalchemy_exception, with_traceback=exc_info[2], from_=e │
│ 2126 │ │ │ │ ) │
│ 2127 │ │ │ else: │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py:208 in raise_ │
│ │
│ 205 │ │ │ exception.__cause__ = replace_context │
│ 206 │ │ │
│ 207 │ │ try: │
│ ❱ 208 │ │ │ raise exception │
│ 209 │ │ finally: │
│ 210 │ │ │ # credit to │
│ 211 │ │ │ # https://cosmicpercolator.com/2016/01/13/exception-leaks-in-python-2-and-3/ │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1900 in │
│ _execute_context │
│ │
│ 1897 │ │ │ │ │ │ │ evt_handled = True │
│ 1898 │ │ │ │ │ │ │ break │
│ 1899 │ │ │ │ if not evt_handled: │
│ ❱ 1900 │ │ │ │ │ self.dialect.do_execute( │
│ 1901 │ │ │ │ │ │ cursor, statement, parameters, context │
│ 1902 │ │ │ │ │ ) │
│ 1903 │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py:736 in do_execute │
│ │
│ 733 │ │ cursor.executemany(statement, parameters) │
│ 734 │ │
│ 735 │ def do_execute(self, cursor, statement, parameters, context=None): │
│ ❱ 736 │ │ cursor.execute(statement, parameters) │
│ 737 │ │
│ 738 │ def do_execute_no_params(self, cursor, statement, context=None): │
│ 739 │ │ cursor.execute(statement) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OperationalError: (sqlite3.OperationalError) no such column: True
[SQL: UPDATE pipeline_build SET contains_code = True]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/agarcia/.local/bin/zenml:8 in <module> │
│ │
│ 5 from zenml.cli.cli import cli │
│ 6 if __name__ == '__main__': │
│ 7 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │
│ ❱ 8 │ sys.exit(cli()) │
│ 9 │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1130 in __call__ │
│ │
│ 1127 │ │
│ 1128 │ def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any: │
│ 1129 │ │ """Alias for :meth:`main`.""" │
│ ❱ 1130 │ │ return self.main(*args, **kwargs) │
│ 1131 │
│ 1132 │
│ 1133 class Command(BaseCommand): │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1055 in main │
│ │
│ 1052 │ │ try: │
│ 1053 │ │ │ try: │
│ 1054 │ │ │ │ with self.make_context(prog_name, args, **extra) as ctx: │
│ ❱ 1055 │ │ │ │ │ rv = self.invoke(ctx) │
│ 1056 │ │ │ │ │ if not standalone_mode: │
│ 1057 │ │ │ │ │ │ return rv │
│ 1058 │ │ │ │ │ # it's not safe to `ctx.exit(rv)` here! │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1657 in invoke │
│ │
│ 1654 │ │ │ │ super().invoke(ctx) │
│ 1655 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx) │
│ 1656 │ │ │ │ with sub_ctx: │
│ ❱ 1657 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │
│ 1658 │ │ │
│ 1659 │ │ # In chain mode we create the contexts step by step, but after the │
│ 1660 │ │ # base command has been invoked. Because at that point we do not │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1404 in invoke │
│ │
│ 1401 │ │ │ echo(style(message, fg="red"), err=True) │
│ 1402 │ │ │
│ 1403 │ │ if self.callback is not None: │
│ ❱ 1404 │ │ │ return ctx.invoke(self.callback, **ctx.params) │
│ 1405 │ │
│ 1406 │ def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]: │
│ 1407 │ │ """Return a list of completions for the incomplete value. Looks │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:760 in invoke │
│ │
│ 757 │ │ │
│ 758 │ │ with augment_usage_errors(__self): │
│ 759 │ │ │ with ctx: │
│ ❱ 760 │ │ │ │ return __callback(*args, **kwargs) │
│ 761 │ │
│ 762 │ def forward( │
│ 763 │ │ __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any # noqa: B902 │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/decorators.py:26 in new_func │
│ │
│ 23 │ """ │
│ 24 │ │
│ 25 │ def new_func(*args, **kwargs): # type: ignore │
│ ❱ 26 │ │ return f(get_current_context(), *args, **kwargs) │
│ 27 │ │
│ 28 │ return update_wrapper(t.cast(F, new_func), f) │
│ 29 │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/cli/base.py:298 in clean │
│ │
│ 295 │ │ │ │ analytics_opt_in=gc.analytics_opt_in, │
│ 296 │ │ │ │ version=gc.version, │
│ 297 │ │ │ ) │
│ ❱ 298 │ │ │ fresh_gc.set_default_store() │
│ 299 │ │ │ declare(f"Reinitialized ZenML global config at {Path.cwd()}.") │
│ 300 │ │
│ 301 │ else: │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/config/global_config.py:619 in │
│ set_default_store │
│ │
│ 616 │ │ │ AnalyticsEvent.INITIALIZED_STORE │
│ 617 │ │ ) as analytics_handler: │
│ 618 │ │ │ default_store_cfg = self.get_default_store() │
│ ❱ 619 │ │ │ self._configure_store(default_store_cfg) │
│ 620 │ │ │ logger.debug("Using the default store for the global config.") │
│ 621 │ │ │ analytics_handler.metadata = { │
│ 622 │ │ │ │ "store_type": default_store_cfg.type.value │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/config/global_config.py:399 in │
│ _configure_store │
│ │
│ 396 │ │ │
│ 397 │ │ # TODO: Revisit the flow regarding the registration of the default │
│ 398 │ │ # entities once the analytics v1 is removed. │
│ ❱ 399 │ │ store = BaseZenStore.create_store(config, True, **kwargs) │
│ 400 │ │ │
│ 401 │ │ logger.debug(f"Configuring the global store to {store.config}") │
│ 402 │ │ self.store = store.config │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/base_zen_store.py:240 in │
│ create_store │
│ │
│ 237 │ │ """ │
│ 238 │ │ logger.debug(f"Creating store with config '{config}'...") │
│ 239 │ │ store_class = BaseZenStore.get_store_class(config.type) │
│ ❱ 240 │ │ store = store_class( │
│ 241 │ │ │ config=config, │
│ 242 │ │ │ skip_default_registrations=skip_default_registrations, │
│ 243 │ │ │ **kwargs, │
│ │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/base_zen_store.py:145 in │
│ __init__ │
│ │
│ 142 │ │ try: │
│ 143 │ │ │ self._initialize() │
│ 144 │ │ except Exception as e: │
│ ❱ 145 │ │ │ raise RuntimeError( │
│ 146 │ │ │ │ f"Error initializing {self.type.value} store with URL " │
│ 147 │ │ │ │ f"'{self.url}': {str(e)}" │
│ 148 │ │ │ ) from e │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Error initializing sql store with URL 'sqlite:////home/agarcia/.config/zenml/local_stores/default_zen_store/zenml.db': (sqlite3.OperationalError) no such column: True
[SQL: UPDATE pipeline_build SET contains_code = True]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
Hi @Galeos93! I was unable to reproduce the error, unfortunately. I'm on MacOS and I ran it in a fresh virtual environment (by way of pyenv
) without any errors. (i.e. the zenml init
ran without issue). I think you're on a linux system, so asked my colleague who also is using a linux system to try (in fresh environment etc) and the steps you described above also worked without issue.
The only thing I can think to check / ask is what virtual environment you're using? If it was something non-standard potentially that might cause some issue with the migration. Otherwise I'm not quite sure what the difference between your system / setup and ours.
Hi again @strickvl. I was trying very hard to fix this but I did not succeed. I set up a clean docker container with python 3.8 and zenml init
worked perfectly. Therefore, I assume there is some (unknown) problem with my system. I will close this :)
Contact Details [Optional]
No response
System Information
zenml, version 0.38.0 Python 3.8.16
What happened?
I get an error on the SQL local store when executing
zenml init
. I expected it to work flawlessly. I executed the code at https://github.com/zenml-io/zenml/blob/main/examples/quickstart/notebooks/quickstart.ipynb.Reproduction steps
Relevant log output
Code of Conduct