search

netbox-community / netbox-docker

🐳 Docker Image of NetBox
https://github.com/netbox-community/netbox-docker/wiki
Apache License 2.0
1.73k stars 800 forks source link

Issue after upgrading to latest version #1249

Closed kguest closed 3 weeks ago

kguest commented 1 month ago

Current Behavior

Ran through the upgrade steps with no errors - I have a backup so can run through this again and provide screenshots if required.

Ran docker compose up - container starts but gets stuck on the health check for the DB.

Added DB_WAIT_DEBUG: 1 to my env variables and will attach the error output below.

Expected Behavior

I expected that due to the upgrade process being successful that the container would run,

Docker Compose Version

Docker Compose version v2.27.0

Docker Version

Client: Docker Engine - Community
 Version:           26.1.3
 API version:       1.45
 Go version:        go1.21.10
 Git commit:        b72abbb
 Built:             Thu May 16 08:33:48 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.3
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.10
  Git commit:       8e96db1
  Built:            Thu May 16 08:33:48 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.32
  GitCommit:        8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

The git Revision

0c99ff8b5663db3e0db5a45660cebda9f917508b

The git Status

On branch release
Your branch is up to date with 'origin/release'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   configuration/plugins.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        Dockerfile-Plugins
        plugin_requirements.txt

no changes added to commit (use "git add" and/or "git commit -a")

Startup Command

sudo docker compose up

NetBox Logs

netbox-1               | Traceback (most recent call last):
netbox-1               |   File "/opt/netbox/netbox/./manage.py", line 10, in <module>
netbox-1               |     execute_from_command_line(sys.argv)
netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
netbox-1               |     utility.execute()
netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
netbox-1               |     self.fetch_command(subcommand).run_from_argv(self.argv)
netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
netbox-1               |     self.execute(*args, **cmd_options)
netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 454, in execute
netbox-1               |     self.check()
netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 486, in check
netbox-1               |     all_issues = checks.run_checks(
netbox-1               |                  ^^^^^^^^^^^^^^^^^^
netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/checks/registry.py", line 88, in run_checks
netbox-1               |     new_errors = check(app_configs=app_configs, databases=databases)
netbox-1               |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/checks/model_checks.py", line 18, in check_all_models
netbox-1               |     models = apps.get_models()
netbox-1               |              ^^^^^^^^^^^^^^^^^
netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/apps/registry.py", line 181, in get_models
netbox-1               |     self.check_models_ready()
netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/apps/registry.py", line 143, in check_models_ready
netbox-1               |     raise AppRegistryNotReady("Models aren't loaded yet.")
netbox-1               | django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.
netbox-1               | ⏳ Waiting on DB... (27s / 30s)
netbox-1               | ❌ Waited 30s or more for the DB to become ready.
dependency failed to start: container netbox-docker-netbox-1 exited (1)

Content of docker-compose.override.yml

version: '3.4'
services:
  netbox:
    ports:
      - "8000:8080"
    environment:
      NETBOX_DELETE_LEGACY_DATA: "true"
      BANNER_LOGIN: '<a href="/api/plugins/sso/login" class="btn btn-primary btn-block">Login with Google SSO</a>'
      ALLOWED_HOSTS: [redacted]
      DB_WAIT_DEBUG: 1
      #MAX_DB_WAIT_TIME: 90
    build:
      context: .
      dockerfile: Dockerfile-Plugins
  netbox-worker:
    build:
      context: .
      dockerfile: Dockerfile-Plugins
  netbox-housekeeping:
    build:
      context: .
      dockerfile: Dockerfile-Plugins
  postgres:
    image: postgres:15-alpine
cimnine commented 1 month ago

It seems you use plugins. Make sure that every plugin is updated and compatible with NetBox 4.x.

kguest commented 1 month ago

So you were right it looks like I had a couple of plugins that weren't compatible with Netbox 4.x

I've removed them from plugins.py and ran a rebuild to stop them being loaded as they weren't really getting used.

However, now I'm recieving a different error:


netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/loader.py", line 276, in build_graph
netbox-docker-netbox-1               |     self.graph.validate_consistency()
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/graph.py", line 198, in validate_consistency
netbox-docker-netbox-1               |     [n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/graph.py", line 198, in <listcomp>
netbox-docker-netbox-1               |     [n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
netbox-docker-netbox-1               |      ^^^^^^^^^^^^^^^
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/graph.py", line 60, in raise_error
netbox-docker-netbox-1               |     raise NodeNotFoundError(self.error_message, self.key, origin=self.origin)
netbox-docker-netbox-1               | django.db.migrations.exceptions.NodeNotFoundError: Migration dcim.0183_devicetype_exclude_from_utilization dependencies reference nonexistent parent node ('dcim', '0182_zero_length_cable_fix')
netbox-docker-netbox-1               | 
netbox-docker-netbox-1               | The above exception was the direct cause of the following exception:
netbox-docker-netbox-1               | 
netbox-docker-netbox-1               | Traceback (most recent call last):
netbox-docker-netbox-1               |   File "/opt/netbox/netbox/./manage.py", line 10, in <module>
netbox-docker-netbox-1               |     execute_from_command_line(sys.argv)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
netbox-docker-netbox-1               |     utility.execute()
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
netbox-docker-netbox-1               |     self.fetch_command(subcommand).run_from_argv(self.argv)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
netbox-docker-netbox-1               |     self.execute(*args, **cmd_options)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
netbox-docker-netbox-1               |     output = self.handle(*args, **options)
netbox-docker-netbox-1               |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/commands/showmigrations.py", line 66, in handle
netbox-docker-netbox-1               |     return self.show_list(connection, options["app_label"])
netbox-docker-netbox-1               |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/commands/showmigrations.py", line 85, in show_list
netbox-docker-netbox-1               |     loader = MigrationLoader(connection, ignore_no_migrations=True)
netbox-docker-netbox-1               |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/loader.py", line 58, in __init__
netbox-docker-netbox-1               |     self.build_graph()
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/loader.py", line 295, in build_graph
netbox-docker-netbox-1               |     raise NodeNotFoundError(
netbox-docker-netbox-1               | django.db.migrations.exceptions.NodeNotFoundError: Migration dcim.0183_devicetype_exclude_from_utilization depends on nonexistent node ('dcim', '0182_zero_length_cable_fix'). Django tried to replace migration dcim.0182_zero_length_cable_fix with any of [dcim.0167_squashed_0182] but wasn't able to because some of the replaced migrations are already applied.
netbox-docker-netbox-1               | ⏳ Waiting on DB... (15s / 30s)```
cimnine commented 1 month ago

This now looks like a problem unrelated to the Docker image, and hence you should use them upstream issue tracker and discussions and chat to get help. Generally, make sure that you are not skipping too many versions while upgrading and that you read the release notes carefully. Afaik, before the upgrade to NB 4.x, you should first upgrade to the latest patch version of NB 3.x

kguest commented 1 month ago

Ah I see, I'll try updating to latest 3.x first!

Cheers for the help.

danielskowronski commented 1 month ago

I had exactly the same issue on v4.0.3 installed on an empty database and the problem has two parts:

  1. I missed collectstatic phase after pip install - obvious user mistake
  2. Netbox appears to be unable to bootstrap with plugins enabled

The second one seems concerning, but is probably more related to Netbox itself, not this repo. However, if Netbox app has that behaviour by design, I think this repo should incorporate some logic to bypass it - the same way as superuser insert is handled.