search

sunscrapers / djoser

REST implementation of Django authentication system.
MIT License
2.55k stars 458 forks source link

`PicklingError: Cannot pickle ResolverMatch` in Django 5.1 for any endpoint that sends an email #842

Closed johnnymetz closed 1 week ago

johnnymetz commented 2 months ago

Background

I recently upgraded to Django 5.1.1 and I'm seeing a PicklingError in my unit tests for all djoser endpoints that send an email.

Here are some minimalist unit tests that are failing (note I'm using pytest-django):

@pytest.mark.django_db
def test_create_user(client):
    url = reverse("user-list")
    data = {
        "email": "jj@email.com",
        "password": "pw123",
        "re_password": "pw123",
    }
    client.post(url, data=data)

@pytest.mark.django_db
def test_activate_user(client):
    u = UserFactory(is_active=False)
    url = reverse("user-activation")
    data = {
        "uid": djoser.utils.encode_uid(u.pk),
        "token": default_token_generator.make_token(u),
    }
    client.post(url, data=data)

@pytest.mark.django_db
def test_reset_password(client):
    u = UserFactory()
    url = reverse("user-reset-password-confirm")
    new_pw = "new_pw"
    data = {
        "uid": djoser.utils.encode_uid(u.pk),
        "token": default_token_generator.make_token(u),
        "new_password": new_pw,
        "re_new_password": new_pw,
    }
    client.post(url, data=data)

All of these tests fail. Here's the full traceback for the first test (test_create_user):

Traceback (most recent call last):
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
    return view_func(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/rest_framework/viewsets.py", line 124, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/rest_framework/mixins.py", line 19, in create
    self.perform_create(serializer)
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/djoser/views.py", line 142, in perform_create
    settings.EMAIL.activation(self.request, context).send(to)
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/templated_mail/mail.py", line 78, in send
    super(BaseEmailMessage, self).send(*args, **kwargs)
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/django/core/mail/message.py", line 301, in send
    return self.get_connection(fail_silently).send_messages([self])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/django/core/mail/backends/locmem.py", line 31, in send_messages
    mail.outbox.append(copy.deepcopy(message))
                       ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py", line 161, in deepcopy
    rv = reductor(4)
         ^^^^^^^^^^^
  File "/Users/johnnymetz/Repos/personal/notifier-app/venv/lib/python3.11/site-packages/django/urls/resolvers.py", line 105, in __reduce_ex__
    raise PicklingError(f"Cannot pickle {self.__class__.__qualname__}.")
_pickle.PicklingError: Cannot pickle ResolverMatch.

users/tests/views/test_user.py:28 (test_create_user)
client = <django.test.client.Client object at 0x104808190>

    @pytest.mark.django_db
    def test_create_user(client):
        url = reverse("user-list")
        data = {
            "email": "jj@email.com",
            "password": TEST_PASSWORD,
            "re_password": TEST_PASSWORD,
        }
>       client.post(url, data=data)

test_user.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../venv/lib/python3.11/site-packages/django/test/client.py:1158: in post
    response = super().post(
../../../venv/lib/python3.11/site-packages/django/test/client.py:503: in post
    return self.generic(
../../../venv/lib/python3.11/site-packages/django/test/client.py:676: in generic
    return self.request(**r)
../../../venv/lib/python3.11/site-packages/django/test/client.py:1092: in request
    self.check_exception(response)
../../../venv/lib/python3.11/site-packages/django/test/client.py:805: in check_exception
    raise exc_value
../../../venv/lib/python3.11/site-packages/django/core/handlers/exception.py:55: in inner
    response = get_response(request)
../../../venv/lib/python3.11/site-packages/django/core/handlers/base.py:197: in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
../../../venv/lib/python3.11/site-packages/django/views/decorators/csrf.py:65: in _view_wrapper
    return view_func(request, *args, **kwargs)
../../../venv/lib/python3.11/site-packages/rest_framework/viewsets.py:124: in view
    return self.dispatch(request, *args, **kwargs)
../../../venv/lib/python3.11/site-packages/rest_framework/views.py:509: in dispatch
    response = self.handle_exception(exc)
../../../venv/lib/python3.11/site-packages/rest_framework/views.py:469: in handle_exception
    self.raise_uncaught_exception(exc)
../../../venv/lib/python3.11/site-packages/rest_framework/views.py:480: in raise_uncaught_exception
    raise exc
../../../venv/lib/python3.11/site-packages/rest_framework/views.py:506: in dispatch
    response = handler(request, *args, **kwargs)
../../../venv/lib/python3.11/site-packages/rest_framework/mixins.py:19: in create
    self.perform_create(serializer)
../../../venv/lib/python3.11/site-packages/djoser/views.py:142: in perform_create
    settings.EMAIL.activation(self.request, context).send(to)
../../../venv/lib/python3.11/site-packages/templated_mail/mail.py:78: in send
    super(BaseEmailMessage, self).send(*args, **kwargs)
../../../venv/lib/python3.11/site-packages/django/core/mail/message.py:301: in send
    return self.get_connection(fail_silently).send_messages([self])
../../../venv/lib/python3.11/site-packages/django/core/mail/backends/locmem.py:31: in send_messages
    mail.outbox.append(copy.deepcopy(message))
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:172: in deepcopy
    y = _reconstruct(x, memo, *rv)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:271: in _reconstruct
    state = deepcopy(state, memo)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:146: in deepcopy
    y = copier(x, memo)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:231: in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:172: in deepcopy
    y = _reconstruct(x, memo, *rv)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:271: in _reconstruct
    state = deepcopy(state, memo)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:146: in deepcopy
    y = copier(x, memo)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:231: in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:172: in deepcopy
    y = _reconstruct(x, memo, *rv)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:271: in _reconstruct
    state = deepcopy(state, memo)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:146: in deepcopy
    y = copier(x, memo)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:231: in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
/opt/homebrew/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/copy.py:161: in deepcopy
    rv = reductor(4)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = ResolverMatch(func=djoser.views.UserViewSet, args=(), kwargs={}, url_name='user-list', app_names=[], namespaces=[], route='api/auth/users/$')
protocol = 4

    def __reduce_ex__(self, protocol):
>       raise PicklingError(f"Cannot pickle {self.__class__.__qualname__}.")
E       _pickle.PicklingError: Cannot pickle ResolverMatch.
AndreMiras commented 3 weeks ago

Got a similar issue while going from Django 5.0.9 to Django 5.1.1 with djoser 2.2.3

tomwojcik commented 1 week ago

Fixed in 2.3.1.