search

netbox-community / netbox

The premier source of truth powering network automation. Open source under Apache 2. Try NetBox Cloud free: https://netboxlabs.com/free-netbox-cloud/
http://netboxlabs.com/oss/netbox/
Apache License 2.0
15.97k stars 2.56k forks source link

Upgrade from 3.5-beta1 to 3.5-beta2 breaks the instance #12289

Closed peteeckel closed 1 year ago

peteeckel commented 1 year ago

NetBox version

3.5-beta2

Python version

3.8

Steps to Reproduce

Upgrade from 3.5-beta1 to 3.5-beta2

Expected Behavior

A working NetBox 3.5-beta2 installation

Observed Behavior

After the upgrade from 3.5-beta1 to 3.5-beta2, the NetBox instance is pretty much unusable. Almost every click into the user ID (both Dashboard widgets and Sidebar items) results in an Exception (see below).

This is obviously caused by a change in the migration `extras/migrations/0085_synced_data.py', which has additional columns in 3.5-beta2 as compared to 3.5-beta1:

diff --git a/netbox/extras/migrations/0085_synced_data.py b/netbox/extras/migrations/0085_synced_data.py
index 4790cd5..3728454 100644
--- a/netbox/extras/migrations/0085_synced_data.py
+++ b/netbox/extras/migrations/0085_synced_data.py
@@ -28,6 +28,11 @@ class Migration(migrations.Migration):
         ),
         migrations.AddField(
             model_name='configcontext',
+            name='auto_sync_enabled',
+            field=models.BooleanField(default=False),
+        ),
+        migrations.AddField(
+            model_name='configcontext',
             name='data_synced',
             field=models.DateTimeField(blank=True, editable=False, null=True),
         ),
@@ -49,6 +54,11 @@ class Migration(migrations.Migration):
         ),
         migrations.AddField(
             model_name='exporttemplate',
+            name='auto_sync_enabled',
+            field=models.BooleanField(default=False),
+        ),
+        migrations.AddField(
+            model_name='exporttemplate',
             name='data_synced',
             field=models.DateTimeField(blank=True, editable=False, null=True),
         ),

Not sure if this qualifies as a real bug, as it is an incompatibility between two beta versions (should this be supported?), but you may run into it while testing. The workaround is to drop and re-create the NetBox database or to create the columns manually.

Environment:

Request Method: GET
Request URL: http://dns.dev.hindenburgring.com/dcim/device-types/

Django Version: 4.1.8
Python Version: 3.8.13
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'corsheaders',
 'debug_toolbar',
 'graphiql_debug_toolbar',
 'django_filters',
 'django_tables2',
 'django_prometheus',
 'graphene_django',
 'mptt',
 'rest_framework',
 'social_django',
 'taggit',
 'timezone_field',
 'core',
 'circuits',
 'dcim',
 'ipam',
 'extras',
 'tenancy',
 'users',
 'utilities',
 'virtualization',
 'wireless',
 'django_rq',
 'drf_spectacular',
 'drf_spectacular_sidecar',
 'netbox_dns.DNSConfig']
Installed Middleware:
['graphiql_debug_toolbar.middleware.DebugToolbarMiddleware',
 'django_prometheus.middleware.PrometheusBeforeMiddleware',
 'corsheaders.middleware.CorsMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.middleware.security.SecurityMiddleware',
 'netbox.middleware.RemoteUserMiddleware',
 'netbox.middleware.CoreMiddleware',
 'django_prometheus.middleware.PrometheusAfterMiddleware']

Template error:
In template /opt/netbox/netbox/utilities/templates/buttons/export.html, error at line 8
   column extras_exporttemplate.auto_sync_enabled does not exist
LINE 1: ...ta_file_id", "extras_exporttemplate"."data_path", "extras_ex...
                                                             ^

   1 : <div class="dropdown">
   2 :   <button type="button" class="btn btn-sm btn-purple dropdown-toggle" data-bs-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
   3 :     <i class="mdi mdi-download"></i>&nbsp;Export
   4 :   </button>
   5 :   <ul class="dropdown-menu dropdown-menu-end">
   6 :     <li><a class="dropdown-item" href="?{% if url_params %}{{ url_params }}&{% endif %}export=table">Current View</a></li>
   7 :     <li><a class="dropdown-item" href="?{% if url_params %}{{ url_params }}&{% endif %}export">All Data ({{ data_format }})</a></li>
   8 :      {% if export_templates %} 
   9 :       <li>
   10 :         <hr class="dropdown-divider">
   11 :       </li>
   12 :       {% for et in export_templates %}
   13 :         <li>
   14 :           <a class="dropdown-item" href="?{% if url_params %}{{ url_params }}&{% endif %}export={{ et.name }}"
   15 :             {% if et.description %} title="{{ et.description }}"{% endif %}
   16 :           >
   17 :             {{ et.name }}
   18 :           </a>

Traceback (most recent call last):
  File "/opt/netbox/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)

The above exception (column extras_exporttemplate.auto_sync_enabled does not exist
LINE 1: ...ta_file_id", "extras_exporttemplate"."data_path", "extras_ex...
                                                             ^
) was the direct cause of the following exception:
  File "/opt/netbox/lib/python3.8/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
  File "/opt/netbox/lib/python3.8/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/netbox/lib/python3.8/site-packages/django/views/generic/base.py", line 103, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/netbox/netbox/netbox/views/generic/base.py", line 77, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/opt/netbox/netbox/utilities/views.py", line 99, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/opt/netbox/lib/python3.8/site-packages/django/views/generic/base.py", line 142, in dispatch
    return handler(request, *args, **kwargs)
  File "/opt/netbox/netbox/netbox/views/generic/bulk_views.py", line 184, in get
    return render(request, self.template_name, context)
  File "/opt/netbox/lib/python3.8/site-packages/django/shortcuts.py", line 24, in render
    content = loader.render_to_string(template_name, context, request, using=using)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/loader.py", line 62, in render_to_string
    return template.render(context, request)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/loader_tags.py", line 63, in render
    result = block.nodelist.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/loader_tags.py", line 63, in render
    result = block.nodelist.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/loader_tags.py", line 63, in render
    result = block.nodelist.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/defaulttags.py", line 321, in render
    return nodelist.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/library.py", line 278, in render
    return t.render(new_context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 177, in render
    return self._render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/opt/netbox/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "/opt/netbox/lib/python3.8/site-packages/django/template/defaulttags.py", line 320, in render
    if match:
  File "/opt/netbox/lib/python3.8/site-packages/django/db/models/query.py", line 408, in __bool__
    self._fetch_all()
  File "/opt/netbox/lib/python3.8/site-packages/django/db/models/query.py", line 1867, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/opt/netbox/lib/python3.8/site-packages/django/db/models/query.py", line 87, in __iter__
    results = compiler.execute_sql(
  File "/opt/netbox/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File "/opt/netbox/lib/python3.8/site-packages/debug_toolbar/panels/sql/tracking.py", line 247, in execute
    return self._record(self.cursor.execute, sql, params)
  File "/opt/netbox/lib/python3.8/site-packages/debug_toolbar/panels/sql/tracking.py", line 168, in _record
    return method(sql, params)
  File "/opt/netbox/lib/python3.8/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
  File "/opt/netbox/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/opt/netbox/lib/python3.8/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/netbox/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/netbox/lib/python3.8/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)

Exception Type: ProgrammingError at /dcim/device-types/
Exception Value: column extras_exporttemplate.auto_sync_enabled does not exist
LINE 1: ...ta_file_id", "extras_exporttemplate"."data_path", "extras_ex...
                                                             ^
jeremystretch commented 1 year ago

Yes, this is expected. You'll need to upgrade from a v3.4 database (or from an empty database) for each beta, as the migrations have been modified. This has always been the case for our beta releases.

peteeckel commented 1 year ago

Thanks, I thought that would be the case but just wanted to be sure.