zenml-io / zenml

ZenML 🙏: The bridge between ML and Ops. https://zenml.io.
https://zenml.io
Apache License 2.0
3.92k stars 427 forks source link

[BUG]: Error in SQL local store when executing zenml init #1493

Closed Galeos93 closed 1 year ago

Galeos93 commented 1 year ago

Contact Details [Optional]

No response

System Information

zenml, version 0.38.0 Python 3.8.16

What happened?

I get an error on the SQL local store when executing zenml init. I expected it to work flawlessly. I executed the code at https://github.com/zenml-io/zenml/blob/main/examples/quickstart/notebooks/quickstart.ipynb.

Reproduction steps

  1. python -m pip install "zenml[server]
  2. zenml integration install sklearn mlflow evidently -y
  3. zenml init ...

Relevant log output

agarcia@agarcia-MS-7A34:~/repos$ zenml init
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1900 in               │
│ _execute_context                                                                                 │
│                                                                                                  │
│   1897 │   │   │   │   │   │   │   evt_handled = True                                            │
│   1898 │   │   │   │   │   │   │   break                                                         │
│   1899 │   │   │   │   if not evt_handled:                                                       │
│ ❱ 1900 │   │   │   │   │   self.dialect.do_execute(                                              │
│   1901 │   │   │   │   │   │   cursor, statement, parameters, context                            │
│   1902 │   │   │   │   │   )                                                                     │
│   1903                                                                                           │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py:736 in do_execute  │
│                                                                                                  │
│    733 │   │   cursor.executemany(statement, parameters)                                         │
│    734 │                                                                                         │
│    735 │   def do_execute(self, cursor, statement, parameters, context=None):                    │
│ ❱  736 │   │   cursor.execute(statement, parameters)                                             │
│    737 │                                                                                         │
│    738 │   def do_execute_no_params(self, cursor, statement, context=None):                      │
│    739 │   │   cursor.execute(statement)                                                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OperationalError: duplicate column name: zenml_version

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/base_zen_store.py:143 in       │
│ __init__                                                                                         │
│                                                                                                  │
│   140 │   │   super().__init__(**kwargs)                                                         │
│   141 │   │                                                                                      │
│   142 │   │   try:                                                                               │
│ ❱ 143 │   │   │   self._initialize()                                                             │
│   144 │   │   except Exception as e:                                                             │
│   145 │   │   │   raise RuntimeError(                                                            │
│   146 │   │   │   │   f"Error initializing {self.type.value} store with URL "                    │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/sql_zen_store.py:840 in        │
│ _initialize                                                                                      │
│                                                                                                  │
│    837 │   │   │   not self.skip_migrations                                                      │
│    838 │   │   │   and ENV_ZENML_DISABLE_DATABASE_MIGRATION not in os.environ                    │
│    839 │   │   ):                                                                                │
│ ❱  840 │   │   │   self.migrate_database()                                                       │
│    841 │                                                                                         │
│    842 │   def _create_mysql_database(                                                           │
│    843 │   │   self,                                                                             │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/sql_zen_store.py:903 in        │
│ migrate_database                                                                                 │
│                                                                                                  │
│    900 │   │   │   │   )                                                                         │
│    901 │   │   │   # Case 3: the database has been migrated with alembic before. Just            │
│    902 │   │   │   # upgrade to the latest revision.                                             │
│ ❱  903 │   │   │   self.alembic.upgrade()                                                        │
│    904 │   │   else:                                                                             │
│    905 │   │   │   if self.alembic.db_is_empty():                                                │
│    906 │   │   │   │   # Case 1: the database is empty. We can just create the                   │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/migrations/alembic.py:201 in   │
│ upgrade                                                                                          │
│                                                                                                  │
│   198 │   │   │   │   revision, rev  # type:ignore [arg-type]                                    │
│   199 │   │   │   )                                                                              │
│   200 │   │                                                                                      │
│ ❱ 201 │   │   self.run_migrations(do_upgrade)                                                    │
│   202 │                                                                                          │
│   203 │   def downgrade(self, revision: str) -> None:                                            │
│   204 │   │   """Revert the database to a previous version.                                      │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/migrations/alembic.py:152 in   │
│ run_migrations                                                                                   │
│                                                                                                  │
│   149 │   │   │   )                                                                              │
│   150 │   │   │                                                                                  │
│   151 │   │   │   with self.environment_context.begin_transaction():                             │
│ ❱ 152 │   │   │   │   self.environment_context.run_migrations()                                  │
│   153 │                                                                                          │
│   154 │   def current_revisions(self) -> List[str]:                                              │
│   155 │   │   """Get the current database revisions.                                             │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/runtime/environment.py:853 in           │
│ run_migrations                                                                                   │
│                                                                                                  │
│   850 │   │   """                                                                                │
│   851 │   │   assert self._migration_context is not None                                         │
│   852 │   │   with Operations.context(self._migration_context):                                  │
│ ❱ 853 │   │   │   self.get_context().run_migrations(**kw)                                        │
│   854 │                                                                                          │
│   855 │   def execute(self, sql, execution_options=None):                                        │
│   856 │   │   """Execute the given SQL using the current change context.                         │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/runtime/migration.py:623 in             │
│ run_migrations                                                                                   │
│                                                                                                  │
│    620 │   │   │   │   │   self.impl.static_output(                                              │
│    621 │   │   │   │   │   │   "-- Running %s" % (step.short_log,)                               │
│    622 │   │   │   │   │   )                                                                     │
│ ❱  623 │   │   │   │   step.migration_fn(**kw)                                                   │
│    624 │   │   │   │                                                                             │
│    625 │   │   │   │   # previously, we wouldn't stamp per migration                             │
│    626 │   │   │   │   # if we were in a transaction, however given the more                     │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/migrations/versions/d26471b6fe │
│ 8f_update_build_filtering.py:40 in upgrade                                                       │
│                                                                                                  │
│   37 │   │   │   │   nullable=True,                                                              │
│   38 │   │   │   )                                                                               │
│   39 │   │   )                                                                                   │
│ ❱ 40 │   │   batch_op.add_column(                                                                │
│   41 │   │   │   sa.Column(                                                                      │
│   42 │   │   │   │   "checksum", sqlmodel.sql.sqltypes.AutoString(), nullable=True               │
│   43 │   │   │   )                                                                               │
│                                                                                                  │
│ /usr/lib/python3.8/contextlib.py:120 in __exit__                                                 │
│                                                                                                  │
│   117 │   def __exit__(self, type, value, traceback):                                            │
│   118 │   │   if type is None:                                                                   │
│   119 │   │   │   try:                                                                           │
│ ❱ 120 │   │   │   │   next(self.gen)                                                             │
│   121 │   │   │   except StopIteration:                                                          │
│   122 │   │   │   │   return False                                                               │
│   123 │   │   │   else:                                                                          │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/operations/base.py:381 in               │
│ batch_alter_table                                                                                │
│                                                                                                  │
│   378 │   │   )                                                                                  │
│   379 │   │   batch_op = BatchOperations(self.migration_context, impl=impl)                      │
│   380 │   │   yield batch_op                                                                     │
│ ❱ 381 │   │   impl.flush()                                                                       │
│   382 │                                                                                          │
│   383 │   def get_context(self):                                                                 │
│   384 │   │   """Return the :class:`.MigrationContext` object that's                             │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/operations/batch.py:111 in flush        │
│                                                                                                  │
│   108 │   │   │   if not should_recreate:                                                        │
│   109 │   │   │   │   for opname, arg, kw in self.batch:                                         │
│   110 │   │   │   │   │   fn = getattr(self.operations.impl, opname)                             │
│ ❱ 111 │   │   │   │   │   fn(*arg, **kw)                                                         │
│   112 │   │   │   else:                                                                          │
│   113 │   │   │   │   if self.naming_convention:                                                 │
│   114 │   │   │   │   │   m1 = MetaData(naming_convention=self.naming_convention)                │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/ddl/impl.py:322 in add_column           │
│                                                                                                  │
│   319 │   │   column: "Column",                                                                  │
│   320 │   │   schema: Optional[Union[str, "quoted_name"]] = None,                                │
│   321 │   ) -> None:                                                                             │
│ ❱ 322 │   │   self._exec(base.AddColumn(table_name, column, schema=schema))                      │
│   323 │                                                                                          │
│   324 │   def drop_column(                                                                       │
│   325 │   │   self,                                                                              │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/ddl/impl.py:195 in _exec                │
│                                                                                                  │
│   192 │   │   │   │   assert isinstance(multiparams, tuple)                                      │
│   193 │   │   │   │   multiparams += (params,)                                                   │
│   194 │   │   │                                                                                  │
│ ❱ 195 │   │   │   return conn.execute(construct, multiparams)                                    │
│   196 │                                                                                          │
│   197 │   def execute(                                                                           │
│   198 │   │   self,                                                                              │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/future/engine.py:280 in execute      │
│                                                                                                  │
│   277 │   │   :return: a :class:`_engine.Result` object.                                         │
│   278 │   │                                                                                      │
│   279 │   │   """                                                                                │
│ ❱ 280 │   │   return self._execute_20(                                                           │
│   281 │   │   │   statement, parameters, execution_options or NO_OPTIONS                         │
│   282 │   │   )                                                                                  │
│   283                                                                                            │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1705 in _execute_20   │
│                                                                                                  │
│   1702 │   │   │   │   exc.ObjectNotExecutableError(statement), replace_context=err              │
│   1703 │   │   │   )                                                                             │
│   1704 │   │   else:                                                                             │
│ ❱ 1705 │   │   │   return meth(self, args_10style, kwargs_10style, execution_options)            │
│   1706 │                                                                                         │
│   1707 │   def exec_driver_sql(                                                                  │
│   1708 │   │   self, statement, parameters=None, execution_options=None                          │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py:80 in                     │
│ _execute_on_connection                                                                           │
│                                                                                                  │
│     77 │   def _execute_on_connection(                                                           │
│     78 │   │   self, connection, multiparams, params, execution_options                          │
│     79 │   ):                                                                                    │
│ ❱   80 │   │   return connection._execute_ddl(                                                   │
│     81 │   │   │   self, multiparams, params, execution_options                                  │
│     82 │   │   )                                                                                 │
│     83                                                                                           │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1472 in _execute_ddl  │
│                                                                                                  │
│   1469 │   │   compiled = ddl.compile(                                                           │
│   1470 │   │   │   dialect=dialect, schema_translate_map=schema_translate_map                    │
│   1471 │   │   )                                                                                 │
│ ❱ 1472 │   │   ret = self._execute_context(                                                      │
│   1473 │   │   │   dialect,                                                                      │
│   1474 │   │   │   dialect.execution_ctx_cls._init_ddl,                                          │
│   1475 │   │   │   compiled,                                                                     │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1943 in               │
│ _execute_context                                                                                 │
│                                                                                                  │
│   1940 │   │   │   │   │   branched.close()                                                      │
│   1941 │   │                                                                                     │
│   1942 │   │   except BaseException as e:                                                        │
│ ❱ 1943 │   │   │   self._handle_dbapi_exception(                                                 │
│   1944 │   │   │   │   e, statement, parameters, cursor, context                                 │
│   1945 │   │   │   )                                                                             │
│   1946                                                                                           │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:2124 in               │
│ _handle_dbapi_exception                                                                          │
│                                                                                                  │
│   2121 │   │   │   if newraise:                                                                  │
│   2122 │   │   │   │   util.raise_(newraise, with_traceback=exc_info[2], from_=e)                │
│   2123 │   │   │   elif should_wrap:                                                             │
│ ❱ 2124 │   │   │   │   util.raise_(                                                              │
│   2125 │   │   │   │   │   sqlalchemy_exception, with_traceback=exc_info[2], from_=e             │
│   2126 │   │   │   │   )                                                                         │
│   2127 │   │   │   else:                                                                         │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py:208 in raise_         │
│                                                                                                  │
│   205 │   │   │   exception.__cause__ = replace_context                                          │
│   206 │   │                                                                                      │
│   207 │   │   try:                                                                               │
│ ❱ 208 │   │   │   raise exception                                                                │
│   209 │   │   finally:                                                                           │
│   210 │   │   │   # credit to                                                                    │
│   211 │   │   │   # https://cosmicpercolator.com/2016/01/13/exception-leaks-in-python-2-and-3/   │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1900 in               │
│ _execute_context                                                                                 │
│                                                                                                  │
│   1897 │   │   │   │   │   │   │   evt_handled = True                                            │
│   1898 │   │   │   │   │   │   │   break                                                         │
│   1899 │   │   │   │   if not evt_handled:                                                       │
│ ❱ 1900 │   │   │   │   │   self.dialect.do_execute(                                              │
│   1901 │   │   │   │   │   │   cursor, statement, parameters, context                            │
│   1902 │   │   │   │   │   )                                                                     │
│   1903                                                                                           │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py:736 in do_execute  │
│                                                                                                  │
│    733 │   │   cursor.executemany(statement, parameters)                                         │
│    734 │                                                                                         │
│    735 │   def do_execute(self, cursor, statement, parameters, context=None):                    │
│ ❱  736 │   │   cursor.execute(statement, parameters)                                             │
│    737 │                                                                                         │
│    738 │   def do_execute_no_params(self, cursor, statement, context=None):                      │
│    739 │   │   cursor.execute(statement)                                                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OperationalError: (sqlite3.OperationalError) duplicate column name: zenml_version
[SQL: ALTER TABLE pipeline_build ADD COLUMN zenml_version VARCHAR]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/agarcia/.local/bin/zenml:8 in <module>                                                     │
│                                                                                                  │
│   5 from zenml.cli.cli import cli                                                                │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1130 in __call__                  │
│                                                                                                  │
│   1127 │                                                                                         │
│   1128 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                           │
│   1129 │   │   """Alias for :meth:`main`."""                                                     │
│ ❱ 1130 │   │   return self.main(*args, **kwargs)                                                 │
│   1131                                                                                           │
│   1132                                                                                           │
│   1133 class Command(BaseCommand):                                                               │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1055 in main                      │
│                                                                                                  │
│   1052 │   │   try:                                                                              │
│   1053 │   │   │   try:                                                                          │
│   1054 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                  │
│ ❱ 1055 │   │   │   │   │   rv = self.invoke(ctx)                                                 │
│   1056 │   │   │   │   │   if not standalone_mode:                                               │
│   1057 │   │   │   │   │   │   return rv                                                         │
│   1058 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                               │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1657 in invoke                    │
│                                                                                                  │
│   1654 │   │   │   │   super().invoke(ctx)                                                       │
│   1655 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                    │
│   1656 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1658 │   │                                                                                     │
│   1659 │   │   # In chain mode we create the contexts step by step, but after the                │
│   1660 │   │   # base command has been invoked.  Because at that point we do not                 │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1404 in invoke                    │
│                                                                                                  │
│   1401 │   │   │   echo(style(message, fg="red"), err=True)                                      │
│   1402 │   │                                                                                     │
│   1403 │   │   if self.callback is not None:                                                     │
│ ❱ 1404 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1405 │                                                                                         │
│   1406 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]:  │
│   1407 │   │   """Return a list of completions for the incomplete value. Looks                   │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:760 in invoke                     │
│                                                                                                  │
│    757 │   │                                                                                     │
│    758 │   │   with augment_usage_errors(__self):                                                │
│    759 │   │   │   with ctx:                                                                     │
│ ❱  760 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    761 │                                                                                         │
│    762 │   def forward(                                                                          │
│    763 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902             │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/cli/base.py:173 in init                   │
│                                                                                                  │
│   170 │                                                                                          │
│   171 │   with console.status(f"Initializing ZenML repository at {path}.\n"):                    │
│   172 │   │   try:                                                                               │
│ ❱ 173 │   │   │   Client.initialize(root=path)                                                   │
│   174 │   │   │   declare(f"ZenML repository initialized at {path}.")                            │
│   175 │   │   except InitializationException as e:                                               │
│   176 │   │   │   declare(f"{e}")                                                                │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/client.py:437 in initialize               │
│                                                                                                  │
│    434 │   │   │   config_directory = str(root / REPOSITORY_DIRECTORY_NAME)                      │
│    435 │   │   │   io_utils.create_dir_recursive_if_not_exists(config_directory)                 │
│    436 │   │   │   # Initialize the repository configuration at the custom path                  │
│ ❱  437 │   │   │   Client(root=root)                                                             │
│    438 │                                                                                         │
│    439 │   @property                                                                             │
│    440 │   def uses_local_configuration(self) -> bool:                                           │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/client.py:246 in __call__                 │
│                                                                                                  │
│    243 │   │   │   Client: The global Client instance.                                           │
│    244 │   │   """                                                                               │
│    245 │   │   if args or kwargs:                                                                │
│ ❱  246 │   │   │   return cast("Client", super().__call__(*args, **kwargs))                      │
│    247 │   │                                                                                     │
│    248 │   │   if not cls._global_client:                                                        │
│    249 │   │   │   cls._global_client = cast(                                                    │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/client.py:297 in __init__                 │
│                                                                                                  │
│    294 │   │   self._root: Optional[Path] = None                                                 │
│    295 │   │   self._config: Optional[ClientConfiguration] = None                                │
│    296 │   │                                                                                     │
│ ❱  297 │   │   self._set_active_root(root)                                                       │
│    298 │                                                                                         │
│    299 │   @classmethod                                                                          │
│    300 │   def get_instance(cls) -> Optional["Client"]:                                          │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/client.py:354 in _set_active_root         │
│                                                                                                  │
│    351 │   │                                                                                     │
│    352 │   │   # Sanitize the client configuration to reflect the current                        │
│    353 │   │   # settings                                                                        │
│ ❱  354 │   │   self._sanitize_config()                                                           │
│    355 │                                                                                         │
│    356 │   def _config_path(self) -> Optional[str]:                                              │
│    357 │   │   """Path to the client configuration file.                                         │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/client.py:377 in _sanitize_config         │
│                                                                                                  │
│    374 │   │   if not self._config:                                                              │
│    375 │   │   │   return                                                                        │
│    376 │   │                                                                                     │
│ ❱  377 │   │   active_workspace, active_stack = self.zen_store.validate_active_config(           │
│    378 │   │   │   self._config.active_workspace_id,                                             │
│    379 │   │   │   self._config.active_stack_id,                                                 │
│    380 │   │   │   config_name="repo",                                                           │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/client.py:562 in zen_store                │
│                                                                                                  │
│    559 │   │   Returns:                                                                          │
│    560 │   │   │   The global zen store.                                                         │
│    561 │   │   """                                                                               │
│ ❱  562 │   │   return GlobalConfiguration().zen_store                                            │
│    563 │                                                                                         │
│    564 │   @property                                                                             │
│    565 │   def root(self) -> Optional[Path]:                                                     │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/config/global_config.py:271 in            │
│ __custom_getattribute__                                                                          │
│                                                                                                  │
│   268 │   │   Returns:                                                                           │
│   269 │   │   │   The attribute value.                                                           │
│   270 │   │   """                                                                                │
│ ❱ 271 │   │   value = super().__getattribute__(key)                                              │
│   272 │   │   if key.startswith("_") or key not in type(self).__fields__:                        │
│   273 │   │   │   return value                                                                   │
│   274                                                                                            │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/config/global_config.py:696 in zen_store  │
│                                                                                                  │
│   693 │   │   │   The current zen store.                                                         │
│   694 │   │   """                                                                                │
│   695 │   │   if not self.store:                                                                 │
│ ❱ 696 │   │   │   self.set_default_store()                                                       │
│   697 │   │   elif self._zen_store is None:                                                      │
│   698 │   │   │   self._configure_store(self.store)                                              │
│   699                                                                                            │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/config/global_config.py:619 in            │
│ set_default_store                                                                                │
│                                                                                                  │
│   616 │   │   │   AnalyticsEvent.INITIALIZED_STORE                                               │
│   617 │   │   ) as analytics_handler:                                                            │
│   618 │   │   │   default_store_cfg = self.get_default_store()                                   │
│ ❱ 619 │   │   │   self._configure_store(default_store_cfg)                                       │
│   620 │   │   │   logger.debug("Using the default store for the global config.")                 │
│   621 │   │   │   analytics_handler.metadata = {                                                 │
│   622 │   │   │   │   "store_type": default_store_cfg.type.value                                 │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/config/global_config.py:399 in            │
│ _configure_store                                                                                 │
│                                                                                                  │
│   396 │   │                                                                                      │
│   397 │   │   # TODO: Revisit the flow regarding the registration of the default                 │
│   398 │   │   #  entities once the analytics v1 is removed.                                      │
│ ❱ 399 │   │   store = BaseZenStore.create_store(config, True, **kwargs)                          │
│   400 │   │                                                                                      │
│   401 │   │   logger.debug(f"Configuring the global store to {store.config}")                    │
│   402 │   │   self.store = store.config                                                          │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/base_zen_store.py:240 in       │
│ create_store                                                                                     │
│                                                                                                  │
│   237 │   │   """                                                                                │
│   238 │   │   logger.debug(f"Creating store with config '{config}'...")                          │
│   239 │   │   store_class = BaseZenStore.get_store_class(config.type)                            │
│ ❱ 240 │   │   store = store_class(                                                               │
│   241 │   │   │   config=config,                                                                 │
│   242 │   │   │   skip_default_registrations=skip_default_registrations,                         │
│   243 │   │   │   **kwargs,                                                                      │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/base_zen_store.py:145 in       │
│ __init__                                                                                         │
│                                                                                                  │
│   142 │   │   try:                                                                               │
│   143 │   │   │   self._initialize()                                                             │
│   144 │   │   except Exception as e:                                                             │
│ ❱ 145 │   │   │   raise RuntimeError(                                                            │
│   146 │   │   │   │   f"Error initializing {self.type.value} store with URL "                    │
│   147 │   │   │   │   f"'{self.url}': {str(e)}"                                                  │
│   148 │   │   │   ) from e                                                                       │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Error initializing sql store with URL 'sqlite:////home/agarcia/.config/zenml/local_stores/default_zen_store/zenml.db': (sqlite3.OperationalError) duplicate column name: zenml_version
[SQL: ALTER TABLE pipeline_build ADD COLUMN zenml_version VARCHAR]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

Code of Conduct

strickvl commented 1 year ago

Hi @Galeos93! Sorry you ran into this issue. From what I can tell from the stack trace it appears that one of our internal database migrations failed midway at some point. If you're not too deep into the weeds of your playing round with ZenML -- and it appears you're at the beginning of your journey since you're running the quickstart -- I'd suggest that the quickest way to fix this is just to get a fresh start with a new database. If you run zenml clean that should take care of that. If you need to do it manually I can explain that too, but let me know if you run into any further issues or require help with doing it manually (instead of through the ZenML CLI).

Galeos93 commented 1 year ago

Hi @strickvl, thanks for the help. Unfortunately, I have tried cleaning everything out and I still get the same error. These are the steps I have followed:

Maybe I am missing some key step? Can you reproduce this?

I have also tried the above steps but using zenml clean before the last two steps. I get a different error when I do zenml clean:

agarcia@agarcia-MS-7A34:~/repos/zenml_demo$ zenml clean
The local ZenML dashboard is not running.
DANGER: This will completely delete all artifacts, metadata and stacks 
ever created during the use of ZenML. Pipelines and stack components running non-
locally will still exist. Please delete those manually. 

Are you sure you want to proceed? [y/n]: y
Deleting 'local_stores' directory from global config.
Deleted global ZenML config from /home/agarcia/.config/zenml.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1900 in               │
│ _execute_context                                                                                 │
│                                                                                                  │
│   1897 │   │   │   │   │   │   │   evt_handled = True                                            │
│   1898 │   │   │   │   │   │   │   break                                                         │
│   1899 │   │   │   │   if not evt_handled:                                                       │
│ ❱ 1900 │   │   │   │   │   self.dialect.do_execute(                                              │
│   1901 │   │   │   │   │   │   cursor, statement, parameters, context                            │
│   1902 │   │   │   │   │   )                                                                     │
│   1903                                                                                           │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py:736 in do_execute  │
│                                                                                                  │
│    733 │   │   cursor.executemany(statement, parameters)                                         │
│    734 │                                                                                         │
│    735 │   def do_execute(self, cursor, statement, parameters, context=None):                    │
│ ❱  736 │   │   cursor.execute(statement, parameters)                                             │
│    737 │                                                                                         │
│    738 │   def do_execute_no_params(self, cursor, statement, context=None):                      │
│    739 │   │   cursor.execute(statement)                                                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OperationalError: no such column: True

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/base_zen_store.py:143 in       │
│ __init__                                                                                         │
│                                                                                                  │
│   140 │   │   super().__init__(**kwargs)                                                         │
│   141 │   │                                                                                      │
│   142 │   │   try:                                                                               │
│ ❱ 143 │   │   │   self._initialize()                                                             │
│   144 │   │   except Exception as e:                                                             │
│   145 │   │   │   raise RuntimeError(                                                            │
│   146 │   │   │   │   f"Error initializing {self.type.value} store with URL "                    │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/sql_zen_store.py:840 in        │
│ _initialize                                                                                      │
│                                                                                                  │
│    837 │   │   │   not self.skip_migrations                                                      │
│    838 │   │   │   and ENV_ZENML_DISABLE_DATABASE_MIGRATION not in os.environ                    │
│    839 │   │   ):                                                                                │
│ ❱  840 │   │   │   self.migrate_database()                                                       │
│    841 │                                                                                         │
│    842 │   def _create_mysql_database(                                                           │
│    843 │   │   self,                                                                             │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/sql_zen_store.py:908 in        │
│ migrate_database                                                                                 │
│                                                                                                  │
│    905 │   │   │   if self.alembic.db_is_empty():                                                │
│    906 │   │   │   │   # Case 1: the database is empty. We can just create the                   │
│    907 │   │   │   │   # tables from scratch with alembic.                                       │
│ ❱  908 │   │   │   │   self.alembic.upgrade()                                                    │
│    909 │   │   │   else:                                                                         │
│    910 │   │   │   │   # Case 2: the database is not empty, but has never been                   │
│    911 │   │   │   │   # migrated with alembic before. We need to create the alembic             │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/migrations/alembic.py:201 in   │
│ upgrade                                                                                          │
│                                                                                                  │
│   198 │   │   │   │   revision, rev  # type:ignore [arg-type]                                    │
│   199 │   │   │   )                                                                              │
│   200 │   │                                                                                      │
│ ❱ 201 │   │   self.run_migrations(do_upgrade)                                                    │
│   202 │                                                                                          │
│   203 │   def downgrade(self, revision: str) -> None:                                            │
│   204 │   │   """Revert the database to a previous version.                                      │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/migrations/alembic.py:152 in   │
│ run_migrations                                                                                   │
│                                                                                                  │
│   149 │   │   │   )                                                                              │
│   150 │   │   │                                                                                  │
│   151 │   │   │   with self.environment_context.begin_transaction():                             │
│ ❱ 152 │   │   │   │   self.environment_context.run_migrations()                                  │
│   153 │                                                                                          │
│   154 │   def current_revisions(self) -> List[str]:                                              │
│   155 │   │   """Get the current database revisions.                                             │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/runtime/environment.py:853 in           │
│ run_migrations                                                                                   │
│                                                                                                  │
│   850 │   │   """                                                                                │
│   851 │   │   assert self._migration_context is not None                                         │
│   852 │   │   with Operations.context(self._migration_context):                                  │
│ ❱ 853 │   │   │   self.get_context().run_migrations(**kw)                                        │
│   854 │                                                                                          │
│   855 │   def execute(self, sql, execution_options=None):                                        │
│   856 │   │   """Execute the given SQL using the current change context.                         │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/runtime/migration.py:623 in             │
│ run_migrations                                                                                   │
│                                                                                                  │
│    620 │   │   │   │   │   self.impl.static_output(                                              │
│    621 │   │   │   │   │   │   "-- Running %s" % (step.short_log,)                               │
│    622 │   │   │   │   │   )                                                                     │
│ ❱  623 │   │   │   │   step.migration_fn(**kw)                                                   │
│    624 │   │   │   │                                                                             │
│    625 │   │   │   │   # previously, we wouldn't stamp per migration                             │
│    626 │   │   │   │   # if we were in a transaction, however given the more                     │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/migrations/versions/d26471b6fe │
│ 8f_update_build_filtering.py:45 in upgrade                                                       │
│                                                                                                  │
│   42 │   │   │   │   "checksum", sqlmodel.sql.sqltypes.AutoString(), nullable=True               │
│   43 │   │   │   )                                                                               │
│   44 │   │   )                                                                                   │
│ ❱ 45 │   op.execute("UPDATE pipeline_build SET contains_code = True")                            │
│   46 │   with op.batch_alter_table("pipeline_build", schema=None) as batch_op:                   │
│   47 │   │   batch_op.alter_column(                                                              │
│   48 │   │   │   "contains_code",                                                                │
│ <string>:8 in execute                                                                            │
│ <string>:3 in execute                                                                            │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/operations/ops.py:2414 in execute       │
│                                                                                                  │
│   2411 │   │    :meth:`sqlalchemy.engine.Connection.execution_options`.                          │
│   2412 │   │   """                                                                               │
│   2413 │   │   op = cls(sqltext, execution_options=execution_options)                            │
│ ❱ 2414 │   │   return operations.invoke(op)                                                      │
│   2415                                                                                           │
│   2416                                                                                           │
│   2417 class OpContainer(MigrateOperation):                                                      │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/operations/base.py:399 in invoke        │
│                                                                                                  │
│   396 │   │   fn = self._to_impl.dispatch(                                                       │
│   397 │   │   │   operation, self.migration_context.impl.__dialect__                             │
│   398 │   │   )                                                                                  │
│ ❱ 399 │   │   return fn(self, operation)                                                         │
│   400 │                                                                                          │
│   401 │   def f(self, name: str) -> "conv":                                                      │
│   402 │   │   """Indicate a string name that has already had a naming convention                 │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/operations/toimpl.py:207 in execute_sql │
│                                                                                                  │
│   204 def execute_sql(                                                                           │
│   205 │   operations: "Operations", operation: "ops.ExecuteSQLOp"                                │
│   206 ) -> None:                                                                                 │
│ ❱ 207 │   operations.migration_context.impl.execute(                                             │
│   208 │   │   operation.sqltext, execution_options=operation.execution_options                   │
│   209 │   )                                                                                      │
│   210                                                                                            │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/ddl/impl.py:202 in execute              │
│                                                                                                  │
│   199 │   │   sql: Union["Update", "TextClause", str],                                           │
│   200 │   │   execution_options: None = None,                                                    │
│   201 │   ) -> None:                                                                             │
│ ❱ 202 │   │   self._exec(sql, execution_options)                                                 │
│   203 │                                                                                          │
│   204 │   def alter_column(                                                                      │
│   205 │   │   self,                                                                              │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/alembic/ddl/impl.py:195 in _exec                │
│                                                                                                  │
│   192 │   │   │   │   assert isinstance(multiparams, tuple)                                      │
│   193 │   │   │   │   multiparams += (params,)                                                   │
│   194 │   │   │                                                                                  │
│ ❱ 195 │   │   │   return conn.execute(construct, multiparams)                                    │
│   196 │                                                                                          │
│   197 │   def execute(                                                                           │
│   198 │   │   self,                                                                              │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/future/engine.py:280 in execute      │
│                                                                                                  │
│   277 │   │   :return: a :class:`_engine.Result` object.                                         │
│   278 │   │                                                                                      │
│   279 │   │   """                                                                                │
│ ❱ 280 │   │   return self._execute_20(                                                           │
│   281 │   │   │   statement, parameters, execution_options or NO_OPTIONS                         │
│   282 │   │   )                                                                                  │
│   283                                                                                            │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1705 in _execute_20   │
│                                                                                                  │
│   1702 │   │   │   │   exc.ObjectNotExecutableError(statement), replace_context=err              │
│   1703 │   │   │   )                                                                             │
│   1704 │   │   else:                                                                             │
│ ❱ 1705 │   │   │   return meth(self, args_10style, kwargs_10style, execution_options)            │
│   1706 │                                                                                         │
│   1707 │   def exec_driver_sql(                                                                  │
│   1708 │   │   self, statement, parameters=None, execution_options=None                          │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py:333 in               │
│ _execute_on_connection                                                                           │
│                                                                                                  │
│    330 │   │   self, connection, multiparams, params, execution_options, _force=False            │
│    331 │   ):                                                                                    │
│    332 │   │   if _force or self.supports_execution:                                             │
│ ❱  333 │   │   │   return connection._execute_clauseelement(                                     │
│    334 │   │   │   │   self, multiparams, params, execution_options                              │
│    335 │   │   │   )                                                                             │
│    336 │   │   else:                                                                             │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1572 in               │
│ _execute_clauseelement                                                                           │
│                                                                                                  │
│   1569 │   │   │   schema_translate_map=schema_translate_map,                                    │
│   1570 │   │   │   linting=self.dialect.compiler_linting | compiler.WARN_LINTING,                │
│   1571 │   │   )                                                                                 │
│ ❱ 1572 │   │   ret = self._execute_context(                                                      │
│   1573 │   │   │   dialect,                                                                      │
│   1574 │   │   │   dialect.execution_ctx_cls._init_compiled,                                     │
│   1575 │   │   │   compiled_sql,                                                                 │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1943 in               │
│ _execute_context                                                                                 │
│                                                                                                  │
│   1940 │   │   │   │   │   branched.close()                                                      │
│   1941 │   │                                                                                     │
│   1942 │   │   except BaseException as e:                                                        │
│ ❱ 1943 │   │   │   self._handle_dbapi_exception(                                                 │
│   1944 │   │   │   │   e, statement, parameters, cursor, context                                 │
│   1945 │   │   │   )                                                                             │
│   1946                                                                                           │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:2124 in               │
│ _handle_dbapi_exception                                                                          │
│                                                                                                  │
│   2121 │   │   │   if newraise:                                                                  │
│   2122 │   │   │   │   util.raise_(newraise, with_traceback=exc_info[2], from_=e)                │
│   2123 │   │   │   elif should_wrap:                                                             │
│ ❱ 2124 │   │   │   │   util.raise_(                                                              │
│   2125 │   │   │   │   │   sqlalchemy_exception, with_traceback=exc_info[2], from_=e             │
│   2126 │   │   │   │   )                                                                         │
│   2127 │   │   │   else:                                                                         │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py:208 in raise_         │
│                                                                                                  │
│   205 │   │   │   exception.__cause__ = replace_context                                          │
│   206 │   │                                                                                      │
│   207 │   │   try:                                                                               │
│ ❱ 208 │   │   │   raise exception                                                                │
│   209 │   │   finally:                                                                           │
│   210 │   │   │   # credit to                                                                    │
│   211 │   │   │   # https://cosmicpercolator.com/2016/01/13/exception-leaks-in-python-2-and-3/   │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1900 in               │
│ _execute_context                                                                                 │
│                                                                                                  │
│   1897 │   │   │   │   │   │   │   evt_handled = True                                            │
│   1898 │   │   │   │   │   │   │   break                                                         │
│   1899 │   │   │   │   if not evt_handled:                                                       │
│ ❱ 1900 │   │   │   │   │   self.dialect.do_execute(                                              │
│   1901 │   │   │   │   │   │   cursor, statement, parameters, context                            │
│   1902 │   │   │   │   │   )                                                                     │
│   1903                                                                                           │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py:736 in do_execute  │
│                                                                                                  │
│    733 │   │   cursor.executemany(statement, parameters)                                         │
│    734 │                                                                                         │
│    735 │   def do_execute(self, cursor, statement, parameters, context=None):                    │
│ ❱  736 │   │   cursor.execute(statement, parameters)                                             │
│    737 │                                                                                         │
│    738 │   def do_execute_no_params(self, cursor, statement, context=None):                      │
│    739 │   │   cursor.execute(statement)                                                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OperationalError: (sqlite3.OperationalError) no such column: True
[SQL: UPDATE pipeline_build SET contains_code = True]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/agarcia/.local/bin/zenml:8 in <module>                                                     │
│                                                                                                  │
│   5 from zenml.cli.cli import cli                                                                │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1130 in __call__                  │
│                                                                                                  │
│   1127 │                                                                                         │
│   1128 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                           │
│   1129 │   │   """Alias for :meth:`main`."""                                                     │
│ ❱ 1130 │   │   return self.main(*args, **kwargs)                                                 │
│   1131                                                                                           │
│   1132                                                                                           │
│   1133 class Command(BaseCommand):                                                               │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1055 in main                      │
│                                                                                                  │
│   1052 │   │   try:                                                                              │
│   1053 │   │   │   try:                                                                          │
│   1054 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                  │
│ ❱ 1055 │   │   │   │   │   rv = self.invoke(ctx)                                                 │
│   1056 │   │   │   │   │   if not standalone_mode:                                               │
│   1057 │   │   │   │   │   │   return rv                                                         │
│   1058 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                               │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1657 in invoke                    │
│                                                                                                  │
│   1654 │   │   │   │   super().invoke(ctx)                                                       │
│   1655 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                    │
│   1656 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1658 │   │                                                                                     │
│   1659 │   │   # In chain mode we create the contexts step by step, but after the                │
│   1660 │   │   # base command has been invoked.  Because at that point we do not                 │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:1404 in invoke                    │
│                                                                                                  │
│   1401 │   │   │   echo(style(message, fg="red"), err=True)                                      │
│   1402 │   │                                                                                     │
│   1403 │   │   if self.callback is not None:                                                     │
│ ❱ 1404 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1405 │                                                                                         │
│   1406 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]:  │
│   1407 │   │   """Return a list of completions for the incomplete value. Looks                   │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/core.py:760 in invoke                     │
│                                                                                                  │
│    757 │   │                                                                                     │
│    758 │   │   with augment_usage_errors(__self):                                                │
│    759 │   │   │   with ctx:                                                                     │
│ ❱  760 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    761 │                                                                                         │
│    762 │   def forward(                                                                          │
│    763 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902             │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/click/decorators.py:26 in new_func              │
│                                                                                                  │
│    23 │   """                                                                                    │
│    24 │                                                                                          │
│    25 │   def new_func(*args, **kwargs):  # type: ignore                                         │
│ ❱  26 │   │   return f(get_current_context(), *args, **kwargs)                                   │
│    27 │                                                                                          │
│    28 │   return update_wrapper(t.cast(F, new_func), f)                                          │
│    29                                                                                            │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/cli/base.py:298 in clean                  │
│                                                                                                  │
│   295 │   │   │   │   analytics_opt_in=gc.analytics_opt_in,                                      │
│   296 │   │   │   │   version=gc.version,                                                        │
│   297 │   │   │   )                                                                              │
│ ❱ 298 │   │   │   fresh_gc.set_default_store()                                                   │
│   299 │   │   │   declare(f"Reinitialized ZenML global config at {Path.cwd()}.")                 │
│   300 │                                                                                          │
│   301 │   else:                                                                                  │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/config/global_config.py:619 in            │
│ set_default_store                                                                                │
│                                                                                                  │
│   616 │   │   │   AnalyticsEvent.INITIALIZED_STORE                                               │
│   617 │   │   ) as analytics_handler:                                                            │
│   618 │   │   │   default_store_cfg = self.get_default_store()                                   │
│ ❱ 619 │   │   │   self._configure_store(default_store_cfg)                                       │
│   620 │   │   │   logger.debug("Using the default store for the global config.")                 │
│   621 │   │   │   analytics_handler.metadata = {                                                 │
│   622 │   │   │   │   "store_type": default_store_cfg.type.value                                 │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/config/global_config.py:399 in            │
│ _configure_store                                                                                 │
│                                                                                                  │
│   396 │   │                                                                                      │
│   397 │   │   # TODO: Revisit the flow regarding the registration of the default                 │
│   398 │   │   #  entities once the analytics v1 is removed.                                      │
│ ❱ 399 │   │   store = BaseZenStore.create_store(config, True, **kwargs)                          │
│   400 │   │                                                                                      │
│   401 │   │   logger.debug(f"Configuring the global store to {store.config}")                    │
│   402 │   │   self.store = store.config                                                          │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/base_zen_store.py:240 in       │
│ create_store                                                                                     │
│                                                                                                  │
│   237 │   │   """                                                                                │
│   238 │   │   logger.debug(f"Creating store with config '{config}'...")                          │
│   239 │   │   store_class = BaseZenStore.get_store_class(config.type)                            │
│ ❱ 240 │   │   store = store_class(                                                               │
│   241 │   │   │   config=config,                                                                 │
│   242 │   │   │   skip_default_registrations=skip_default_registrations,                         │
│   243 │   │   │   **kwargs,                                                                      │
│                                                                                                  │
│ /home/agarcia/.local/lib/python3.8/site-packages/zenml/zen_stores/base_zen_store.py:145 in       │
│ __init__                                                                                         │
│                                                                                                  │
│   142 │   │   try:                                                                               │
│   143 │   │   │   self._initialize()                                                             │
│   144 │   │   except Exception as e:                                                             │
│ ❱ 145 │   │   │   raise RuntimeError(                                                            │
│   146 │   │   │   │   f"Error initializing {self.type.value} store with URL "                    │
│   147 │   │   │   │   f"'{self.url}': {str(e)}"                                                  │
│   148 │   │   │   ) from e                                                                       │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Error initializing sql store with URL 'sqlite:////home/agarcia/.config/zenml/local_stores/default_zen_store/zenml.db': (sqlite3.OperationalError) no such column: True
[SQL: UPDATE pipeline_build SET contains_code = True]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
strickvl commented 1 year ago

Hi @Galeos93! I was unable to reproduce the error, unfortunately. I'm on MacOS and I ran it in a fresh virtual environment (by way of pyenv) without any errors. (i.e. the zenml init ran without issue). I think you're on a linux system, so asked my colleague who also is using a linux system to try (in fresh environment etc) and the steps you described above also worked without issue.

The only thing I can think to check / ask is what virtual environment you're using? If it was something non-standard potentially that might cause some issue with the migration. Otherwise I'm not quite sure what the difference between your system / setup and ours.

Galeos93 commented 1 year ago

Hi again @strickvl. I was trying very hard to fix this but I did not succeed. I set up a clean docker container with python 3.8 and zenml init worked perfectly. Therefore, I assume there is some (unknown) problem with my system. I will close this :)