search

inventree / InvenTree

Open Source Inventory Management System
https://docs.inventree.org
MIT License
4.27k stars 772 forks source link

Docker Compose - inventree-server crashing due to missing django_q_schedule table? #8103

Closed victorhooi closed 1 month ago

victorhooi commented 1 month ago

Deployment Method

Describe the problem*

I am trying to deploy InvenTree locally using the provided Docker compose file at:

https://github.com/inventree/InvenTree/blob/master/contrib/container/docker-compose.yml

I have placed the following three files in a single directory:

I then run:

docker compose pull
docker compose up -d

(At first, I got hit with a gotcha around the Caddyfile - it wasn't mentioned this was needed - and when Docker compose tried to spin everything up, it hit this error:

❯ docker compose up --detach
[+] Running 4/5
 ✔ Container inventree-cache   Running                                                                                                                                                                                                                                    0.0s 
 ✔ Container inventree-db      Running                                                                                                                                                                                                                                    0.0s 
 ✔ Container inventree-worker  Running                                                                                                                                                                                                                                    0.0s 
 ✔ Container inventree-server  Started                                                                                                                                                                                                                                    0.0s 
 ⠙ Container inventree-proxy   Starting                                                                                                                                                                                                                                   0.1s 
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/Users/bytedance/inventree-docker/Caddyfile" to rootfs at "/etc/caddy/Caddyfile": mount /Users/bytedance/inventree-docker/Caddyfile:/etc/caddy/Caddyfile (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

(It seems at least one other person hit this - https://github.com/inventree/InvenTree/discussions/7401 - it might be worth documenting it - or figuring out why the Docker compose file doesn't fail gracefully here?)

Anyway, clearing that error - the next issue is that inventree-server and inventree-worker are both crashing.

(container logs output is below under "Relevant log output").

Steps to Reproduce

  1. Download the following files, and place them into a directory
  2. cd to that directory
  3. Run docker compose pull
  4. Run docker compose up -d
  5. Run 'docker ps -ato see the status of the InvenTree containers. Notice thatinventree-serverandinventree-worker` are crashing
  6. Run docker logs inventree-server to see the issuedpc

Relevant log output

Container logs for inventree-server:

LINE 1: SELECT 1 AS "a" FROM "django_q_schedule" WHERE "django_q_sch...
                             ^
Traceback (most recent call last):
  File "/root/.local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
psycopg.errors.UndefinedTable: relation "django_q_schedule" does not exist
LINE 1: ...", "django_q_schedule"."intended_date_kwarg" FROM "django_q_...
                                                             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.local/bin/gunicorn", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/root/.local/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 67, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]", prog=prog).run()
  File "/root/.local/lib/python3.11/site-packages/gunicorn/app/base.py", line 236, in run
    super().run()
  File "/root/.local/lib/python3.11/site-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
    ^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 58, in __init__
    self.setup(app)
  File "/root/.local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 118, in setup
    self.app.wsgi()
  File "/root/.local/lib/python3.11/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
                    ^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
           ^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/gunicorn/util.py", line 371, in import_app
    mod = importlib.import_module(module)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/inventree/src/backend/InvenTree/InvenTree/wsgi.py", line 17, in <module>
    application = get_wsgi_application()  # pragma: no cover
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
    django.setup(set_prefix=False)
  File "/root/.local/lib/python3.11/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/root/.local/lib/python3.11/site-packages/django/apps/registry.py", line 124, in populate
    app_config.ready()
  File "/home/inventree/src/backend/InvenTree/InvenTree/apps.py", line 57, in ready
    self.start_background_tasks()
  File "/home/inventree/src/backend/InvenTree/InvenTree/apps.py", line 116, in start_background_tasks
    for existing_task in Schedule.objects.all():
  File "/root/.local/lib/python3.11/site-packages/django/db/models/query.py", line 398, in __iter__
    self._fetch_all()
  File "/root/.local/lib/python3.11/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
    cursor.execute(sql, params)
  File "/root/.local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/root/.local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/root/.local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
django.db.utils.ProgrammingError: relation "django_q_schedule" does not exist
LINE 1: ...", "django_q_schedule"."intended_date_kwarg" FROM "django_q_...
                                                             ^

Container logs for `inventree-worker`:

❯ docker logs inventree-worker
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
Loading config file : /home/inventree/data/config.yaml
No idea what 'int.worker' is!
victorhooi commented 1 month ago

Hmm, looking at the postgres logs, I see this:

❯ docker logs inventree-db
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data/pgdb ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
syncing data to disk ... ok

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data/pgdb -l logfile start

waiting for server to start....2024-09-10 05:53:53.672 UTC [49] LOG:  starting PostgreSQL 13.16 (Debian 13.16-1.pgdg120+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2024-09-10 05:53:53.674 UTC [49] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-09-10 05:53:53.680 UTC [50] LOG:  database system was shut down at 2024-09-10 05:53:53 UTC
2024-09-10 05:53:53.684 UTC [49] LOG:  database system is ready to accept connections
 done
server started
CREATE DATABASE

/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*

2024-09-10 05:53:53.990 UTC [49] LOG:  received fast shutdown request
waiting for server to shut down....2024-09-10 05:53:53.992 UTC [49] LOG:  aborting any active transactions
2024-09-10 05:53:53.993 UTC [49] LOG:  background worker "logical replication launcher" (PID 56) exited with exit code 1
2024-09-10 05:53:53.993 UTC [51] LOG:  shutting down
2024-09-10 05:53:54.003 UTC [49] LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

2024-09-10 05:53:54.103 UTC [1] LOG:  starting PostgreSQL 13.16 (Debian 13.16-1.pgdg120+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2024-09-10 05:53:54.103 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2024-09-10 05:53:54.103 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2024-09-10 05:53:54.104 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-09-10 05:53:54.108 UTC [64] LOG:  database system was shut down at 2024-09-10 05:53:53 UTC
2024-09-10 05:53:54.111 UTC [1] LOG:  database system is ready to accept connections
2024-09-10 05:53:54.552 UTC [71] ERROR:  relation "django_content_type" does not exist at character 106
2024-09-10 05:53:54.552 UTC [71] STATEMENT:  SELECT "django_content_type"."id", "django_content_type"."app_label", "django_content_type"."model" FROM "django_content_type"
2024-09-10 05:53:54.564 UTC [71] ERROR:  relation "common_inventreesetting" does not exist at character 112
2024-09-10 05:53:54.564 UTC [71] STATEMENT:  SELECT "common_inventreesetting"."id", "common_inventreesetting"."value", "common_inventreesetting"."key" FROM "common_inventreesetting" WHERE UPPER("common_inventreesetting"."key"::text) = UPPER('CURRENCY_CODES') ORDER BY "common_inventreesetting"."id" ASC LIMIT 1
2024-09-10 05:53:54.581 UTC [71] ERROR:  relation "common_inventreesetting" does not exist at character 112
2024-09-10 05:53:54.581 UTC [71] STATEMENT:  SELECT "common_inventreesetting"."id", "common_inventreesetting"."value", "common_inventreesetting"."key" FROM "common_inventreesetting" WHERE UPPER("common_inventreesetting"."key"::text) = UPPER('CURRENCY_CODES') ORDER BY "common_inventreesetting"."id" ASC LIMIT 1
2024-09-10 05:53:54.581 UTC [71] ERROR:  relation "common_inventreesetting" does not exist at character 112
2024-09-10 05:53:54.581 UTC [71] STATEMENT:  SELECT "common_inventreesetting"."id", "common_inventreesetting"."value", "common_inventreesetting"."key" FROM "common_inventreesetting" WHERE UPPER("common_inventreesetting"."key"::text) = UPPER('CURRENCY_CODES') ORDER BY "common_inventreesetting"."id" ASC LIMIT 1
2024-09-10 05:53:54.583 UTC [71] ERROR:  relation "common_inventreesetting" does not exist at character 112
2024-09-10 05:53:54.583 UTC [71] STATEMENT:  SELECT "common_inventreesetting"."id", "common_inventreesetting"."value", "common_inventreesetting"."key" FROM "common_inventreesetting" WHERE UPPER("common_inventreesetting"."key"::text) = UPPER('CURRENCY_CODES') ORDER BY "common_inventreesetting"."id" ASC LIMIT 1
2024-09-10 05:53:54.583 UTC [71] ERROR:  relation "common_inventreesetting" does not exist at character 112
2024-09-10 05:53:54.583 UTC [71] STATEMENT:  SELECT "common_inventreesetting"."id", "common_inventreesetting"."value", "common_inventreesetting"."key" FROM "common_inventreesetting" WHERE UPPER("common_inventreesetting"."key"::text) = UPPER('CURRENCY_CODES') ORDER BY "common_inventreesetting"."id" ASC LIMIT 1
2024-09-10 05:53:54.584 UTC [71] ERROR:  relation "common_inventreesetting" does not exist at character 112
2024-09-10 05:53:54.584 UTC [71] STATEMENT:  SELECT "common_inventreesetting"."id", "common_inventreesetting"."value", "common_inventreesetting"."key" FROM "common_inventreesetting" WHERE UPPER("common_inventreesetting"."key"::text) = UPPER('CURRENCY_CODES') ORDER BY "common_inventreesetting"."id" ASC LIMIT 1
2024-09-10 05:53:54.586 UTC [71] ERROR:  relation "common_inventreesetting" does not exist at character 112
matmair commented 1 month ago

I recommend reading the docs https://docs.inventree.org/en/stable/start/docker_install/#initial-database-setup

SchrodingersGat commented 1 month ago

In particular:

https://docs.inventree.org/en/stable/start/docker_install/#update-database

jdguillot commented 1 month ago

I performed both initial database setup and a update database on a fresh install. My server container was starting fine but the worker was restarting with the same error messages. I was able to fix it by adjusting the command in the docker-compose file for the worker container.

Config from the most recent main branch

# InvenTree/contrib/container/docker-compose.yml
inventree-worker:
        # If you wish to specify a particular InvenTree version, do so here
        image: inventree/inventree:${INVENTREE_TAG:-stable}
        container_name: inventree-worker
        command: invoke int.worker
        depends_on:
            - inventree-server
        env_file:
            - .env
        volumes:
            # Data volume must map to /home/inventree/data
            - ${INVENTREE_EXT_VOLUME}:/home/inventree/data:z
        restart: unless-stopped

Fixed command

# InvenTree/contrib/container/docker-compose.yml
inventree-worker:
        # If you wish to specify a particular InvenTree version, do so here
        image: inventree/inventree:${INVENTREE_TAG:-stable}
        container_name: inventree-worker
        command: invoke worker
        depends_on:
            - inventree-server
        env_file:
            - .env
        volumes:
            # Data volume must map to /home/inventree/data
            - ${INVENTREE_EXT_VOLUME}:/home/inventree/data:z
        restart: unless-stopped
SchrodingersGat commented 1 month ago

Ah good catch. @matmair would you be on board with reverting just the worker command as there will be many existing setups which will be affected by this change

SchrodingersGat commented 1 month ago

@victorhooi this has now been fixed in #8126 - please update and check