search

zeratax / matrix-registration

a token based matrix registration api
https://zeratax.github.io/matrix-registration/
MIT License
226 stars 41 forks source link

Database initialization during startup fails #73

Open spantaleev opened 3 years ago

spantaleev commented 3 years ago

I'm trying the 0.9.1 Docker image.

Configuration is like this:

admin_api_shared_secret: MySecret
allow_cors: false
base_url: /matrix-registration
client_logo: static/images/element-logo.png
client_redirect: https://element.DOMAIN
db: postgresql://matrix_registration:password@matrix-postgres:5432/matrix_registration
host: 0.0.0.0
logging:
    disable_existing_loggers: false
    formatters:
        brief:
            format: '%(name)s - %(levelname)s - %(message)s'
        precise:
            format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
    handlers:
        console:
            class: logging.StreamHandler
            formatter: brief
            level: INFO
            stream: ext://sys.stdout
    root:
        handlers:
        - console
        level: DEBUG
    version: 1
password:
    min_length: 8
port: 5000
rate_limit:
- 100 per day
- 10 per minute
registration_shared_secret: test
server_location: http://matrix-nginx-proxy:12080
server_name: DOMAIN

The matrix_registration Postgres database is empty.

I start matrix-registration with the following command: docker run -it --rm --name matrix-registration --log-driver=none --user=991:991 --cap-drop=ALL --network=matrix --mount type=bind,src=/matrix/matrix-registration/config,dst=/config,ro --mount type=bind,src=/matrix/matrix-registration/data,dst=/data docker.io/zeratax/matrix-registration:v0.9.1 --config-path=/config/config.yaml serve

.. which results in:

Traceback (most recent call last):
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.InvalidForeignKey: there is no unique constraint matching given keys for referenced table "ips"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/nix/store/5q2rmk3i4cvjzb5x6s19s5gmv34gjpf6-python3.8-matrix-registration/bin/.matrix-registration-wrapped", line 9, in <module>
    sys.exit(cli())
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/nix/store/7qbdawsyw1bvldh51z1bh1qnb8llz18r-python3.8-Flask-1.1.2/lib/python3.8/site-packages/flask/cli.py", line 586, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 1256, in invoke
    Command.invoke(self, ctx)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/nix/store/5q2rmk3i4cvjzb5x6s19s5gmv34gjpf6-python3.8-matrix-registration/lib/python3.8/site-packages/matrix_registration/app.py", line 43, in cli
    db.create_all()
  File "/nix/store/chxdhhm13lzalclk28xn33xv21ssl1dn-python3.8-Flask-SQLAlchemy-2.4.4/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 1039, in create_all
    self._execute_for_all_tables(app, bind, 'create_all')
  File "/nix/store/chxdhhm13lzalclk28xn33xv21ssl1dn-python3.8-Flask-SQLAlchemy-2.4.4/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 1031, in _execute_for_all_tables
    op(bind=self.get_engine(app, bind), **extra)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 4555, in create_all
    bind._run_visitor(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2097, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1656, in _run_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
    return meth(obj, **kw)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 783, in visit_metadata
    self.traverse_single(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
    return meth(obj, **kw)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 827, in visit_table
    self.connection.execute(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1068, in _execute_ddl
    ret = self._execute_context(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InvalidForeignKey) there is no unique constraint matching given keys for referenced table "ips"

[SQL: 
CREATE TABLE association (
        ips VARCHAR NOT NULL, 
        tokens INTEGER NOT NULL, 
        PRIMARY KEY (ips, tokens), 
        FOREIGN KEY(ips) REFERENCES ips (address), 
        FOREIGN KEY(tokens) REFERENCES tokens (name)
)

]
(Background on this error at: http://sqlalche.me/e/13/f405)

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1208

zeratax commented 3 years ago

yeah this was already reported in #67, but the current fix in 1.0.0.dev7 isnt particularly pretty, but I havent yet had time to get around that. it would work though.

I should also probably backport it since 1.0.0 might still need a little more time

spantaleev commented 3 years ago

Ah, I see! Let me know if you do backport it. We might wait for 1.0 as well.

ghost commented 1 year ago

Im also experiencing this issue. I downloaded the docker image from here.

docker logs:

  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.InvalidForeignKey: there is no unique constraint matching given keys for referenced table "ips"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/nix/store/5q2rmk3i4cvjzb5x6s19s5gmv34gjpf6-python3.8-matrix-registration/bin/.matrix-registration-wrapped", line 9, in <module>
    sys.exit(cli())
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/nix/store/7qbdawsyw1bvldh51z1bh1qnb8llz18r-python3.8-Flask-1.1.2/lib/python3.8/site-packages/flask/cli.py", line 586, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 1256, in invoke
    Command.invoke(self, ctx)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/nix/store/walvddccd81dayrap7lk9krhwlzgbvhz-python3.8-click-7.1.2/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/nix/store/5q2rmk3i4cvjzb5x6s19s5gmv34gjpf6-python3.8-matrix-registration/lib/python3.8/site-packages/matrix_registration/app.py", line 43, in cli
    db.create_all()
  File "/nix/store/chxdhhm13lzalclk28xn33xv21ssl1dn-python3.8-Flask-SQLAlchemy-2.4.4/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 1039, in create_all
    self._execute_for_all_tables(app, bind, 'create_all')
  File "/nix/store/chxdhhm13lzalclk28xn33xv21ssl1dn-python3.8-Flask-SQLAlchemy-2.4.4/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 1031, in _execute_for_all_tables
    op(bind=self.get_engine(app, bind), **extra)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 4555, in create_all
    bind._run_visitor(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2097, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1656, in _run_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
    return meth(obj, **kw)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 783, in visit_metadata
    self.traverse_single(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
    return meth(obj, **kw)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 827, in visit_table
    self.connection.execute(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1068, in _execute_ddl
    ret = self._execute_context(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InvalidForeignKey) there is no unique constraint matching given keys for referenced table "ips"

[SQL: 
CREATE TABLE association (
        ips VARCHAR NOT NULL, 
        tokens INTEGER NOT NULL, 
        PRIMARY KEY (ips, tokens), 
        FOREIGN KEY(ips) REFERENCES ips (address), 
        FOREIGN KEY(tokens) REFERENCES tokens (name)
)

]

config.yaml: db: 'postgresql://synapse:password@db/synapse'

Any updates on this? Why is it happening?

trendsetter37 commented 1 year ago

@t1jlt have you tried using sqlite database first? I noticed this here only expects sqlite for now.

https://github.com/zeratax/matrix-registration/blob/master/alembic/env.py#L79-L84