search

peteeckel / netbox-plugin-dns

NetBox DNS is a NetBox plugin for managing DNS data.
https://pypi.org/project/netbox-plugin-dns
MIT License
195 stars 17 forks source link

Installation manage.py migrate -> relation "django_content_type" does not exist #218

Closed phlpr closed 6 months ago

phlpr commented 6 months ago

Versions NetBox Version: 3.7.4 NetBox DNS Version: 0.22.5 Python Version: 3.9.18

Describe the bug When i install netbox 3.7.4 with netbox-dns 0.22.5 in a fresh environment, i get the following exception when running migrate (/opt/netbox/netbox/manage.py migrate)


Traceback (most recent call last):
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
psycopg.errors.UndefinedTable: relation "django_content_type" does not exist
LINE 1: ..."."app_label", "django_content_type"."model" FROM "django_co...
                                                             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/apps/registry.py", line 124, in populate
    app_config.ready()
  File "/opt/netbox/venv/lib64/python3.9/site-packages/netbox_dns/__init__.py", line 56, in ready
    objtype = ContentType.objects.get_for_model(IPAddress)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/contrib/contenttypes/models.py", line 52, in get_for_model
    ct = self.get(app_label=opts.app_label, model=opts.model_name)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/models/query.py", line 633, in get
    num = len(clone)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/models/query.py", line 380, in __len__
    self._fetch_all()
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
django.db.utils.ProgrammingError: relation "django_content_type" does not exist
LINE 1: ..."."app_label", "django_content_type"."model" FROM "django_co...
```                                                             ^

**To Reproduce**
Steps to reproduce the behavior:
1. Empty environment -> in our case AlmaLinux
2. Go through the installation guide 
3. Run "/opt/netbox/netbox/manage.py migrate"
4. See exception above

Additional info:
- when I deactivate the netbox-dns plugin in configuration.py, migrate runs without errors
- when I activate netbox-dns plugin again (after the first migration is done) and run migrate again, everything works fine
- when i install netbox-dns on a running netbox, migrate runs without errors
- to me it seems like a combination of an "empty database" and "netbox-dns"
- Netbox-Version 3.7.2 and netbox-dns 0.22.4 worked without problems
- I also tried the permissions mentioned in #104 

**Expected result**
Migrations running without any exceptions

**Actual result**
Exception when running migrate

Thank you in advance!
BR
Philipp
peteeckel commented 6 months ago

Hi @phlpr, thanks for raising this!

The CI tests - which don't fail - in GitHub do exactly that: Install the latest version of NetBox and NetBox DNS and then run the migration - with one exception:

What happens when you disable feature_ipam_coupling in configuration.py and run the migration without it?

There appears to have been change in NetBox that affected the code that runs when the get_for_model() method is called, and obviously that code doesn't work properly if the migrations for NetBox haven't been performed yet. As such, it is strictly speaking a NetBox issue. The code in NetBox DNS that causes the exception hasn't been changed for a while.

However, the whole ready() method in __init__.py will be gone when NetBox 4 comes around later this year anyway, which will solve that problem at the root. Strictly speaking it's not really useful at all, it just issues a warning no-one will likely ever see if the custom fields for IPAM Coupling haven't been set up., and so I can take it out immediately - just please check if disabling IPAM Coupling before starting the migration makes it go away.

wranders commented 6 months ago

@peteeckel Thanks for addressing this. I've been experiencing this while trying to develop a deployment for a k8s cluster.

Do you know of a way (command) to perform Netbox migrations with plugins disabled? What I'm trying to figure out if there's a way for a k8s initContainer to run the base migrations before running migrations with plugins enabled. That would alleviate this issue before v4 comes around.

Right now, I'm deploying the application then re-configuring with plugins enabled. It works, but seems pretty kludge-y.

phlpr commented 6 months ago

Hi @peteeckel, thank you for your fast and detailed response.

You are absolutely right, this only happens when the feature_ipam_coupling flag is enabled.

My solution until we switch to v4 (in our deploy-script):

I am happy with this workaround, even if its not the best solution, as @wranders mentioned :)

BR Philipp

peteeckel commented 6 months ago

Released 0.22.6 as an immediate fix for this problem ... sorry!

phlpr commented 6 months ago

Released 0.22.6 as an immediate fix for this problem ... sorry!

Thank you very much! I already tested it, it works fine!