search

mother-of-all-self-hosting / mash-playbook

🐋 Ansible playbook which helps you host various FOSS services as Docker containers on your own server
GNU Affero General Public License v3.0
464 stars 60 forks source link

postgres databases missinig for tandoor and paperless-ngx #281

Open ZenKyma opened 1 week ago

ZenKyma commented 1 week ago

Installing tandoor and paperless-ngx with just install-all leaves them hanging without any postgres database:

sudo journalctl -xeu mash-paperless
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     self.execute(*args, **cmd_options)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     output = self.handle(*args, **options)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 106, in wrapper
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     res = handle_func(*args, **kwargs)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 117, in handle
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     executor = MigrationExecutor(connection, self.migration_progress_callback)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 18, in __init__
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     self.loader = MigrationLoader(self.connection)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/migrations/loader.py", line 58, in __init__
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     self.build_graph()
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/migrations/loader.py", line 235, in build_graph
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     self.applied_migrations = recorder.applied_migrations()
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/migrations/recorder.py", line 81, in applied_migrations
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     if self.has_table():
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:        ^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/migrations/recorder.py", line 57, in has_table
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     with self.connection.cursor() as cursor:
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:          ^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     return func(*args, **kwargs)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:            ^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 330, in cursor
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     return self._cursor()
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:            ^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 306, in _cursor
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     self.ensure_connection()
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     return func(*args, **kwargs)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:            ^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 288, in ensure_connection
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     with self.wrap_database_errors:
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     raise dj_exc_value.with_traceback(traceback) from exc_value
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 289, in ensure_connection
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     self.connect()
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     return func(*args, **kwargs)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:            ^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 270, in connect
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     self.connection = self.get_new_connection(conn_params)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     return func(*args, **kwargs)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:            ^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 275, in get_new_connection
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     connection = self.Database.connect(**conn_params)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:   File "/usr/local/lib/python3.11/site-packages/psycopg/connection.py", line 119, in connect
Oct 07 21:46:52 mysticserver mash-paperless[3356622]:     raise last_ex.with_traceback(None)
Oct 07 21:46:52 mysticserver mash-paperless[3356622]: django.db.utils.OperationalError: connection failed: connection to server at "172.21.0.2", port 5432 failed: FATAL:  database "paperless" does not exist
Oct 07 21:46:53 mysticserver systemd[1]: mash-paperless.service: Main process exited, code=exited, status=1/FAILURE
sudo journalctl -xeu mash-tandoor-api
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     utility.execute()
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     self.fetch_command(subcommand).run_from_argv(self.argv)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/core/management/base.py", line 412, in run_from_argv
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     self.execute(*args, **cmd_options)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/core/management/base.py", line 458, in execute
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     output = self.handle(*args, **options)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/core/management/base.py", line 106, in wrapper
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     res = handle_func(*args, **kwargs)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/core/management/commands/migrate.py", line 117, in handle
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     executor = MigrationExecutor(connection, self.migration_progress_callback)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 18, in __init__
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     self.loader = MigrationLoader(self.connection)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/migrations/loader.py", line 58, in __init__
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     self.build_graph()
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/migrations/loader.py", line 235, in build_graph
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     self.applied_migrations = recorder.applied_migrations()
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/migrations/recorder.py", line 81, in applied_migrations
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     if self.has_table():
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:        ^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/migrations/recorder.py", line 57, in has_table
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     with self.connection.cursor() as cursor:
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:          ^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/utils/asyncio.py", line 26, in inner
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     return func(*args, **kwargs)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:            ^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/backends/base/base.py", line 330, in cursor
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     return self._cursor()
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:            ^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/backends/base/base.py", line 306, in _cursor
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     self.ensure_connection()
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/utils/asyncio.py", line 26, in inner
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     return func(*args, **kwargs)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:            ^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/backends/base/base.py", line 288, in ensure_connection
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     with self.wrap_database_errors:
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     raise dj_exc_value.with_traceback(traceback) from exc_value
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/backends/base/base.py", line 289, in ensure_connection
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     self.connect()
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/utils/asyncio.py", line 26, in inner
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     return func(*args, **kwargs)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:            ^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/backends/base/base.py", line 270, in connect
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     self.connection = self.get_new_connection(conn_params)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/utils/asyncio.py", line 26, in inner
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     return func(*args, **kwargs)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:            ^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/django/db/backends/postgresql/base.py", line 275, in get_new_connection
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     connection = self.Database.connect(**conn_params)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:   File "/opt/recipes/venv/lib/python3.12/site-packages/psycopg2/__init__.py", line 122, in connect
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 07 21:44:23 mysticserver mash-tandoor-api[3344663]: django.db.utils.OperationalError: connection to server at "mash-postgres" (172.21.0.2), port 5432 failed: FATAL:  database "tandoor" does not exist
Oct 07 21:44:24 mysticserver mash-tandoor-api[3344663]: Collecting static files, this may take a while...
Oct 07 21:44:27 mysticserver mash-tandoor-api[3344663]: js-reverse file written to /opt/recipes/cookbook/static/django_js_reverse
Oct 07 21:44:40 mysticserver mash-tandoor-api[3344663]: 1 static file copied to '/opt/recipes/staticfiles', 575 unmodified, 1186 post-processed.
Oct 07 21:44:40 mysticserver mash-tandoor-api[3344663]: Done
Oct 07 21:44:41 mysticserver mash-tandoor-api[3344663]: [2024-10-07 20:44:41 +0000] [1] [INFO] Starting gunicorn 22.0.0
Oct 07 21:44:41 mysticserver mash-tandoor-api[3344663]: [2024-10-07 20:44:41 +0000] [1] [INFO] Listening at: http://[::]:8080 (1)
Oct 07 21:44:41 mysticserver mash-tandoor-api[3344663]: [2024-10-07 20:44:41 +0000] [1] [INFO] Using worker: gthread
Oct 07 21:44:41 mysticserver mash-tandoor-api[3344663]: [2024-10-07 20:44:41 +0000] [15] [INFO] Booting worker with pid: 15
Oct 07 21:44:41 mysticserver mash-tandoor-api[3344663]: [2024-10-07 20:44:41 +0000] [16] [INFO] Booting worker with pid: 16
Oct 07 21:44:41 mysticserver mash-tandoor-api[3344663]: [2024-10-07 20:44:41 +0000] [17] [INFO] Booting worker with pid: 17

even though ansible should have prepared them:

TASK [galaxy/postgres : Create managed database initialization SQL file for paperless] ****************************************************************************************************************************************************************************************
changed: [synkosmos.org]

TASK [galaxy/postgres : Execute Postgres managed database initialization SQL file for paperless] ******************************************************************************************************************************************************************************
changed: [synkosmos.org]

TASK [galaxy/postgres : Delete managed database initialization SQL file for paperless] ****************************************************************************************************************************************************************************************
changed: [synkosmos.org]

TASK [galaxy/postgres : Fail if managed database data appears invalid] ********************************************************************************************************************************************************************************************************
skipping: [synkosmos.org] => (item=name)
skipping: [synkosmos.org] => (item=username)
skipping: [synkosmos.org] => (item=password)

TASK [galaxy/postgres : Create managed database initialization SQL file for tandoor] ******************************************************************************************************************************************************************************************
changed: [synkosmos.org]

TASK [galaxy/postgres : Execute Postgres managed database initialization SQL file for tandoor] ********************************************************************************************************************************************************************************
changed: [synkosmos.org]

TASK [galaxy/postgres : Delete managed database initialization SQL file for tandoor] ******************************************************************************************************************************************************************************************
changed: [synkosmos.org]

TASK [galaxy/postgres : Fail if managed database data appears invalid] ********************************************************************************************************************************************************************************************************
skipping: [synkosmos.org] => (item=name)
skipping: [synkosmos.org] => (item=username)
skipping: [synkosmos.org] => (item=password)
ZenKyma commented 1 week ago

roles have been created:

main-# \du
                                List of roles
   Role name    |                         Attributes
----------------+------------------------------------------------------------
 forgejo        |
 gitea          |
 mash-funkwhale |
 mash-semaphore |
 nextcloud      |
 paperless      |
 postgres       | Superuser, Create role, Create DB, Replication, Bypass RLS
 root           | Superuser, Create role, Create DB, Replication, Bypass RLS
 tandoor        |