install.sh failing - Githubissues

sumansid commented 3 years ago

I am trying to install Dispatch on MacOs and ./install.sh fails at this step :

kevgliss commented 3 years ago

What version of MacOS are you running?

sumansid commented 3 years ago

@kevgliss I'm running macOS Big Sur : 11.5.1

sumansid commented 3 years ago

npm i -f

Fixed the problem and the build was complete. I am however, getting an error : alembic.util.exc.CommandError: Can't locate revision identified by '03f742366750'

kevgliss commented 3 years ago

I think you're facing an issue similar to: Netflix/dispatch#1485 please checkout the solution there and let me know that solves your issue. I hope to create a new release tomorrow that will mitigate this issue. In the meantime you could modify your install.sh to point to the master branch.

sumansid commented 3 years ago

I'm still getting a similar error message even after changing the endpoint to point to the master branch :

alembic.util.exc.CommandError: Can't locate revision identified by '8f364cf49a23'

corylwtsn commented 3 years ago

I am also seeing this issue and I have edited my install script to point at master instead of latest.

Running standard database migrations...
Creating dispatch_core_1 ... done
Creating dispatch_web_run ... done
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 171, in _catch_revision_errors
    yield
  File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 365, in _upgrade_revs
    revs = list(revs)
  File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 706, in iterate_revisions
    revisions, heads = fn(
  File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 1290, in _collect_upgrade_revisions
    current_revisions = self.get_revisions(lower)
  File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 457, in get_revisions
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
  File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 457, in <listcomp>
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
  File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 479, in get_revisions
    return tuple(
  File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 480, in <genexpr>
    self._revision_for_ident(rev_id, branch_label)
  File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 546, in _revision_for_ident
    raise ResolutionError(
alembic.script.revision.ResolutionError: No such revision or branch '8f364cf49a23'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/dispatch", line 8, in <module>
    sys.exit(entrypoint())
  File "/usr/local/lib/python3.8/site-packages/dispatch/cli.py", line 720, in entrypoint
    dispatch_cli()
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/dispatch/cli.py", line 395, in upgrade_database
    alembic_command.upgrade(alembic_cfg, revision, sql=sql, tag=tag)
  File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 294, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 490, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/dispatch/database/revisions/core/env.py", line 70, in <module>
    run_migrations_online()
  File "/usr/local/lib/python3.8/site-packages/dispatch/database/revisions/core/env.py", line 64, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 549, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 283, in upgrade
    return script._upgrade_revs(revision, rev)
  File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 366, in _upgrade_revs
    return [
  File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 203, in _catch_revision_errors
    compat.raise_(util.CommandError(resolution), from_=re)
  File "/usr/local/lib/python3.8/site-packages/alembic/util/compat.py", line 296, in raise_
    raise exception
alembic.util.exc.CommandError: Can't locate revision identified by '8f364cf49a23'

corylwtsn commented 3 years ago

It looks like this error is happening because this revision does not exist on the latest tag of dispatch. https://github.com/Netflix/dispatch/tree/latest/src/dispatch/database/revisions/core/versions

kevgliss commented 3 years ago

I just cut a new release would you mind trying again?

corylwtsn commented 3 years ago

That error has been resolved but now I am seeing a duplicate column error:

Example data loaded. Navigate to /register and create a new user.
Running standard database migrations...
Creating dispatch_core_1 ... done
Creating dispatch_web_run ... done
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.DuplicateColumn: column "slug" of relation "organization" already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/dispatch", line 8, in <module>
    sys.exit(entrypoint())
  File "/usr/local/lib/python3.8/site-packages/dispatch/cli.py", line 715, in entrypoint
    dispatch_cli()
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/dispatch/cli.py", line 390, in upgrade_database
    alembic_command.upgrade(alembic_cfg, revision, sql=sql, tag=tag)
  File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 294, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 490, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/dispatch/database/revisions/core/env.py", line 70, in <module>
    run_migrations_online()
  File "/usr/local/lib/python3.8/site-packages/dispatch/database/revisions/core/env.py", line 64, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 561, in run_migrations
    step.migration_fn(**kw)
  File "/usr/local/lib/python3.8/site-packages/dispatch/database/revisions/core/versions/2021-07-22_c0bc938b058e.py", line 22, in upgrade
    op.add_column("organization", Column("slug", String()), schema="dispatch_core")
  File "<string>", line 8, in add_column
  File "<string>", line 3, in add_column
  File "/usr/local/lib/python3.8/site-packages/alembic/operations/ops.py", line 1801, in add_column
    return operations.invoke(op)
  File "/usr/local/lib/python3.8/site-packages/alembic/operations/base.py", line 354, in invoke
    return fn(self, operation)
  File "/usr/local/lib/python3.8/site-packages/alembic/operations/toimpl.py", line 135, in add_column
    operations.impl.add_column(table_name, column, schema=schema, **kw)
  File "/usr/local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 256, in add_column
    self._exec(base.AddColumn(table_name, column, schema=schema))
  File "/usr/local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 146, in _exec
    return conn.execute(construct, multiparams)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1068, in _execute_ddl
    ret = self._execute_context(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.DuplicateColumn) column "slug" of relation "organization" already exists

[SQL: ALTER TABLE dispatch_core.organization ADD COLUMN slug VARCHAR]
(Background on this error at: http://sqlalche.me/e/13/f405)

vikassinha2019 commented 3 years ago

+1 getting column "slug" of relation "organization" already exists error

kevgliss commented 3 years ago

Thanks for bearing with me here, I guess my local db was in a bit of a weird state when I created that example data set. I've just released https://github.com/Netflix/dispatch/releases/tag/v20210804.1 and ensured the latest tag points to it.

corylwtsn commented 3 years ago

Yep, this seems to work now. Thanks so much for the help @kevgliss!

sumansid commented 3 years ago

Awesome ! Thanks.

radityasurya commented 3 years ago

It seems the problem still persists when you skip loading the example data

Setting up database...
Do you want to load example data (WARNING: this will remove all existing database data) (y/N)?N
Running standard database migrations...
Creating dispatch_core_1 ... done
Creating dispatch_web_run ... done
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedTable: relation "dispatch_user_organization" does not exist

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/dispatch", line 8, in <module>
    sys.exit(entrypoint())
  File "/usr/local/lib/python3.8/site-packages/dispatch/cli.py", line 740, in entrypoint
    dispatch_cli()
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/dispatch/cli.py", line 386, in upgrade_database
    alembic_command.upgrade(alembic_cfg, revision, sql=sql, tag=tag)
  File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 294, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 490, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/dispatch/database/revisions/core/env.py", line 70, in <module>
    run_migrations_online()
  File "/usr/local/lib/python3.8/site-packages/dispatch/database/revisions/core/env.py", line 64, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 561, in run_migrations
    step.migration_fn(**kw)
  File "/usr/local/lib/python3.8/site-packages/dispatch/database/revisions/core/versions/2021-06-15_0ab4f8f54bfa.py", line 21, in upgrade
    op.alter_column(
  File "<string>", line 8, in alter_column
  File "<string>", line 3, in alter_column
  File "/usr/local/lib/python3.8/site-packages/alembic/operations/ops.py", line 1655, in alter_column
    return operations.invoke(alt)
  File "/usr/local/lib/python3.8/site-packages/alembic/operations/base.py", line 354, in invoke
    return fn(self, operation)
  File "/usr/local/lib/python3.8/site-packages/alembic/operations/toimpl.py", line 43, in alter_column
    operations.impl.alter_column(
  File "/usr/local/lib/python3.8/site-packages/alembic/ddl/postgresql.py", line 141, in alter_column
    super(PostgresqlImpl, self).alter_column(
  File "/usr/local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 175, in alter_column
    self._exec(
  File "/usr/local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 146, in _exec
    return conn.execute(construct, multiparams)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1068, in _execute_ddl
    ret = self._execute_context(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "dispatch_user_organization" does not exist

[SQL: ALTER TABLE dispatch_user_organization ALTER COLUMN dispatch_user_id SET NOT NULL]
(Background on this error at: http://sqlalche.me/e/13/f405)
ERROR: 1
Cleaning up...

Netflix / dispatch-docker

install.sh failing #115