jazzband / django-fernet-encrypted-fields

MIT License
42 stars 9 forks source link

Error saving model in django admin #4

Closed victorouttes closed 2 years ago

victorouttes commented 2 years ago

I'm getting this error when saving data using django admin:

Traceback (most recent call last):
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 622, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/views/decorators/cache.py", line 56, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/sites.py", line 236, in inner
    return view(request, *args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1670, in add_view
    return self.changeform_view(request, None, form_url, extra_context)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1549, in changeform_view
    return self._changeform_view(request, object_id, form_url, extra_context)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1593, in _changeform_view
    form_validated = form.is_valid()
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/forms/forms.py", line 190, in is_valid
    return self.is_bound and not self.errors
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/forms/forms.py", line 185, in errors
    self.full_clean()
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/forms/forms.py", line 406, in full_clean
    self._post_clean()
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/forms/models.py", line 411, in _post_clean
    self.instance.full_clean(exclude=exclude, validate_unique=False)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/db/models/base.py", line 1233, in full_clean
    self.clean_fields(exclude=exclude)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/db/models/base.py", line 1275, in clean_fields
    setattr(self, f.attname, f.clean(raw_value, self))
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/db/models/fields/__init__.py", line 670, in clean
    value = self.to_python(value)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/encrypted_fields/fields.py", line 45, in to_python
    value = self.f.decrypt(bytes(value, 'utf-8')).decode('utf-8')
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/cryptography/fernet.py", line 76, in decrypt
    timestamp, data = Fernet._get_unverified_token_data(token)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/cryptography/fernet.py", line 108, in _get_unverified_token_data
    raise InvalidToken
cryptography.fernet.InvalidToken

Model:

from django.db import models
from encrypted_fields.fields import EncryptedCharField

class Setting(models.Model):
    key = models.CharField(max_length=255, verbose_name='Chave', unique=True)
    value = EncryptedCharField(max_length=2000, verbose_name='Valor')

Admin:

from django.contrib import admin
from settings.models.setting import Setting

@admin.register(Setting)
class SettingAdmin(admin.ModelAdmin):
    pass

Django version: 4.0 Database: sqlite

StevenMapes commented 2 years ago

What format is your token? The InvalidToken is not being raise by this package but it being raised by the main pyca cryptography package - https://github.com/pyca/cryptography/blob/main/src/cryptography/fernet.py

If you look at that package the line raising the validation is this

        if not data or data[0] != 0x80:
            raise InvalidToken
victorouttes commented 2 years ago

When I submit a form in Django Admin, it calls default "clean()" function which is:

  def clean(self, value, model_instance):
      """
      Convert the value's type and run validation. Validation errors
      from to_python() and validate() are propagated. Return the correct
      value if no error is raised.
      """
      value = self.to_python(value)
      self.validate(value, model_instance)
      self.run_validators(value)
      return value

The problem is in self.to_python(value). Variable value is the input from form (can be any text by app user).

This call self.to_python(value) is going to encrypted_fields/fields.py:

    def to_python(self, value):
        if value is None or not isinstance(value, str):
            return value
        value = self.f.decrypt(bytes(value, 'utf-8')).decode('utf-8')
        return super(EncryptedFieldMixin, self).to_python(value)

raising error in value = self.f.decrypt(bytes(value, 'utf-8')).decode('utf-8').

So, any Django Admin form input (like "Hello World!") is going to be decrypted in self.f.decrypt(bytes('Hello World!', 'utf-8')). That's the error cause, because decrypt is expecting an encrypted string.

The encrypted_fields to_python(self, value) may be overriding Django's (from django.db.models.fields) to_python(self, value)? Any tip on this?

Thanks in advance.

StevenMapes commented 2 years ago

Here's how I've added a work around for my project

  1. Create a custom Admin class for the model you are using and set that class against the form property of the ModelAdmin or inline form
  2. With the custom form you've created import the EncryptedFieldMixin and overload the _get_validation_exclusions method with the following:
    def _get_validation_exclusions(self):
        """Overload this method to add the encrypted fields to the list of exclusions used by full_clean"""
        exclude = super()._get_validation_exclusions()
        for field in self.fields:
            if isinstance(getattr(self._meta.model, field).field, EncryptedFieldMixin):
                exclude.append(field)
        return exclude

This will add any Encrypted fields on the model that you are using into the exclusion list created by the _post_clean method which then calls the full_clean method on the instance.

You can still add custom validation to the clean_* methods on the form to validate and raise any errors.

I need to check but it may be possible to move this into the EncryptedFieldMixin class against it's full_clean method as well but I have not tested that far down yet.

Here's an example where my model UserProfile has two encrypted fields

admin.py

class UserProfileInline(admin.StackedInline):
    model = UserProfile
    form = UserProfileAdminForm

forms.py

from encrypted_fields.fields import EncryptedFieldMixin

class UserProfileAdminForm(ModelForm):
    class Meta:
        model = UserProfile

    def _get_validation_exclusions(self):
        """Overload this method to add the encrypted fields to the list of exclusions used by full_clean"""
        exclude = super()._get_validation_exclusions()
        for field in self.fields:
            if isinstance(getattr(self._meta.model, field).field, EncryptedFieldMixin):
                exclude.append(field)
        return exclude
StevenMapes commented 2 years ago

Okay I tested the model level and this can be fixed by overloading the clean method at the base model field level by simply running the actions of django.db.models.fields.init.clean without calling the to_python

    def clean(self, value, model_instance):
        """
        Convert the value's type and run validation. Validation errors
        from to_python() and validate() are propagated. Return the correct
        value if no error is raised.
        """
        self.validate(value, model_instance)
        self.run_validators(value)
        return value

However a better method, IMO, would be to use a temporary instance property which we create within the clean method, then call the super, then remove it from the instance. This means the to_python method can be updated to check for that property and, if it exists, can skip decryption. It's more code but means it's less likely to break with future changes to Django's field's clean method as we know it will run as it's suppose to.

    def to_python(self, value):
        if value is None or not isinstance(value, str) or hasattr(self, '_already_decrypted'):
            return value
        value = self.f.decrypt(bytes(value, 'utf-8')).decode('utf-8')
        return super(EncryptedFieldMixin, self).to_python(value)

    def clean(self, value, model_instance):
        """
        Convert the value's type and run validation. Validation errors
        from to_python() and validate() are propagated. Return the correct
        value if no error is raised.
        """
        self._already_decrypted = True
        ret = super().clean(value, model_instance)
        del self._already_decrypted
        return ret

I'll fork and create a new PR based on the latter