NetBox 2.10.3 - Server Error "ObjectChange.user" must be a "User"

[dsg-anthony]

Environment

• Python version: 3.6.8
• NetBox version: 2.10.3

Steps to Reproduce

(GUI and API via Ansible module)

1. Create VM with 1 NIC
2. Create a IPv4 Address
3. Assign IPv4 address to NIC
4. Set IPv4 to primary IP on VM

- name: set primary IP4
     netbox.netbox.netbox_virtual_machine:
        netbox_url: "{{url}}"
        netbox_token: "{{token}}"
        validate_certs: no
   data:
     name: "{{server_name}}"
     primary_ip4: "{{IP}}"

Expected Behavior

IPv4 to be set as primary IP

Observed Behavior

1 out of every 5 or so request will error out. We have noticed other calls failing with the same error but this action is the most common that we have seen fail. This if after we upgraded to 2.10.3 Error:

<class 'ValueError'>

Cannot assign "<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7f0ea5454da0>>": "ObjectChange.user" must be a "User" instance.

Python version: 3.6.8
NetBox version: 2.10.3

[DanSheps]

Can you provide the full backtrace with either debug on or from the email it sends?

[DanSheps]

I am unable to reproduce this error as well. Are you using any plugins?

[dsg-anthony]

We have no plugins installed.

I think this is what you are looking for?

Environment:

Request Method: POST
Request URL: https://np.netbox.###.###/api/dcim/interfaces/

Django Version: 3.1.3
Python Version: 3.6.8
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'cacheops',
 'corsheaders',
 'debug_toolbar',
 'django_filters',
 'django_tables2',
 'django_prometheus',
 'mptt',
 'rest_framework',
 'taggit',
 'timezone_field',
 'circuits',
 'dcim',
 'ipam',
 'extras',
 'secrets',
 'tenancy',
 'users',
 'utilities',
 'virtualization',
 'django_rq',
 'drf_yasg']
Installed Middleware:
['debug_toolbar.middleware.DebugToolbarMiddleware',
 'django_prometheus.middleware.PrometheusBeforeMiddleware',
 'corsheaders.middleware.CorsMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.middleware.security.SecurityMiddleware',
 'netbox.middleware.ExceptionHandlingMiddleware',
 'netbox.middleware.RemoteUserMiddleware',
 'netbox.middleware.LoginRequiredMiddleware',
 'netbox.middleware.APIVersionMiddleware',
 'netbox.middleware.ObjectChangeMiddleware',
 'django_prometheus.middleware.PrometheusAfterMiddleware']

Traceback (most recent call last):
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py\", line 47, in inner
    response = get_response(request)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/core/handlers/base.py\", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py\", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/rest_framework/viewsets.py\", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File \"netbox/netbox/api/views.py\", line 184, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/rest_framework/views.py\", line 509, in dispatch
    response = self.handle_exception(exc)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/rest_framework/views.py\", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/rest_framework/views.py\", line 480, in raise_uncaught_exception
    raise exc
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/rest_framework/views.py\", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/rest_framework/mixins.py\", line 19, in create
    self.perform_create(serializer)
  File \"netbox/netbox/api/views.py\", line 219, in perform_create
    instance = serializer.save()
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/rest_framework/serializers.py\", line 205, in save
    self.instance = self.create(validated_data)
  File \"netbox/extras/api/serializers.py\", line 81, in create
    instance = super().create(validated_data)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/rest_framework/serializers.py\", line 939, in create
    instance = ModelClass._default_manager.create(**validated_data)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/db/models/manager.py\", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/db/models/query.py\", line 447, in create
    obj.save(force_insert=True, using=self.db)
  File \"netbox/dcim/models/device_components.py\", line 479, in save
    return super().save(*args, **kwargs)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/db/models/base.py\", line 754, in save
    force_update=force_update, update_fields=update_fields)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/db/models/base.py\", line 803, in save_base
    update_fields=update_fields, raw=raw, using=using,
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/dispatch/dispatcher.py\", line 179, in send
    for receiver in self._live_receivers(sender)
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/dispatch/dispatcher.py\", line 179, in <listcomp>
    for receiver in self._live_receivers(sender)
  File \"netbox/utilities/utils.py\", line 283, in _curried
    return _curried_func(*args, *moreargs, **{**kwargs, **morekwargs})
  File \"netbox/extras/signals.py\", line 39, in _handle_changed_object
    objectchange.user = request.user
  File \"/opt/netbox-2.10.3/venv/lib64/python3.6/site-packages/django/db/models/fields/related_descriptors.py\", line 220, in __set__
    self.field.remote_field.model._meta.object_name,

Exception Type: ValueError at /api/dcim/interfaces/
Exception Value: Cannot assign "<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7f29b80d2860>>": "ObjectChange.user" must be a "User" instance.

[proudbro]

Related isssue: https://github.com/netbox-community/netbox/issues/5142 The problem has not been resolved and appears itself when bulk objects are changed (no matter via API or WebUI) while other users are working in Netbox.

[jeremystretch]

Looks like a duplicate of #5142. Unfortunately, no one capable of reproducing the reported behavior has yet proposed a fix for it. Please comment on #5142 if you'd like to propose a fix.

[dsg-anthony]

@jeremystretch after some troubleshooting we found out that the cause of this error was having threading enabled in our uWSGI configuration.

If we set the thread count to anything above 1 we would see the error. Increasing the process count > 1 does not cause the error.

[uwsgi]
http = 127.0.0.1:8001
chdir = /opt/netbox
pythonpath = netbox/
wsgi-file = netbox/netbox/wsgi.py
processes = 1
enable-threads = true
threads = 10
master = true

[Dimaqa]

Setting threads=1 in gunicorn.py also fixed this issue for me

[gmurman770]

Hi, everyone! I'm experiencing the same behavior after updating to the 2.10.6.

I making PATCH "virtualization_virtual-machines_partial_update" requests via aionetbox just updating the statuses of various VMs. The statuses are actually being updated, but I'm getting these strange exceptions and 5XX.

/opt/netbox/logs/netbox.log:

Traceback (most recent call last):
  File "/opt/netbox/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/netbox/netbox/netbox/api/views.py", line 184, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/opt/netbox/venv/lib64/python3.6/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/rest_framework/mixins.py", line 82, in partial_update
    return self.update(request, *args, **kwargs)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/rest_framework/mixins.py", line 68, in update
    self.perform_update(serializer)
  File "/opt/netbox/netbox/netbox/api/views.py", line 232, in perform_update
    instance = serializer.save()
  File "/opt/netbox/venv/lib64/python3.6/site-packages/rest_framework/serializers.py", line 200, in save
    self.instance = self.update(self.instance, validated_data)
  File "/opt/netbox/netbox/extras/api/serializers.py", line 93, in update
    instance = super().update(instance, validated_data)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/rest_framework/serializers.py", line 983, in update
    instance.save()
  File "/opt/netbox/venv/lib64/python3.6/site-packages/django/db/models/base.py", line 754, in save
    force_update=force_update, update_fields=update_fields)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/django/db/models/base.py", line 803, in save_base
    update_fields=update_fields, raw=raw, using=using,
  File "/opt/netbox/venv/lib64/python3.6/site-packages/django/dispatch/dispatcher.py", line 179, in send
    for receiver in self._live_receivers(sender)
  File "/opt/netbox/venv/lib64/python3.6/site-packages/django/dispatch/dispatcher.py", line 179, in <listcomp>
    for receiver in self._live_receivers(sender)
  File "/opt/netbox/netbox/utilities/utils.py", line 283, in _curried
    return _curried_func(*args, *moreargs, **{**kwargs, **morekwargs})
  File "/opt/netbox/netbox/extras/signals.py", line 39, in _handle_changed_object
    objectchange.user = request.user
  File "/opt/netbox/venv/lib64/python3.6/site-packages/django/db/models/fields/related_descriptors.py", line 220, in __set__
    self.field.remote_field.model._meta.object_name,
ValueError: Cannot assign "<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fdfdd373128>>": "ObjectChange.user" must be a "User" instance.

Setting threads=1 in gunicorn.py also fixed this issue for me

@Dimaqa does this solution have any impact on performance ??

[Dimaqa]

@gmurman770 I've ran some tests suggested in https://github.com/netbox-community/netbox/issues/5142#issuecomment-726363506 and for some reason they showed even better time results