Open akifd opened 5 years ago
@User3759685 Not sure this can actually be implemented. Due to the encryption, all Django-PGCrypto-Fields are stored in the DB as text
fields (unlimited character length) including Date, DateTime, etc. The JSON field provided by Django is persisted at the DB as a jsonb
field.
You could store encrypted JSON as a text
field but you would lose all operators at the database level to search it (__has_key
, etc.). Currently, there is nothing stopping you from storing JSON in a TextPGPSymmetricalKeyField
and decoding the JSON into a native Python data types.
I question the usefulness of adding direct JSON support in this library. Please let us know if you want to work on a PR for this feature.
I agree it's probably not worth integrating directly, but I needed the basic convenience of json encoding/decoding on top of an encrypted text field (to make code run seamless whether I decide in the end to encrypt the field or not - adapting perhaps only field queries which may be "jsonb aware" or simply raw "text contains" - which latter btw can be very slow over encrypted data).
So here's a gist (JSONTextFieldMixin class based on what Django JSONField does, without the "KeyTransform" gubbins): https://gist.github.com/december1981/ba2a3bb281f7fd4428a178f1d228e68b
Looks like there might be a solution in the form of a JSONTextFieldMixin class that extends the functionality of TextPGPSymmetricalKeyField to support JSON encoding/decoding. However, if you're looking for an alternative approach, you could consider using a custom database field that extends JSONField and adds encryption functionality.
Here's an example implementation:
from django.db import models
from django.contrib.postgres.fields import JSONField
from pgcrypto_fields import PGPTextField
class EncryptedJSONField(PGPTextField, JSONField):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.validators = []
def from_db_value(self, value, expression, connection):
if value is None:
return value
return super().from_db_value(value, expression, connection)
def to_python(self, value):
if value is None:
return value
return super().to_python(value)
def get_prep_value(self, value):
if value is None:
return value
return super().get_prep_value(value)
In this implementation, EncryptedJSONField extends both PGPTextField and JSONField. It overrides the from_db_value, to_python, and get_prep_value methods to ensure that the encrypted JSON data is properly handled by both the PGPTextField and JSONField functionality.
To achieve this, you can create a custom JSONPGPSymmetricalKeyField by subclassing Django's TextField and Django-PGCrypto-Fields' PGPSymmetricalKeyField. Here's a code example:
import json
from django.contrib.postgres.fields import JSONField
from pgcrypto.fields import TextPGPSymmetricalKeyField
class JSONPGPSymmetricalKeyField(JSONField, TextPGPSymmetricalKeyField):
def from_db_value(self, value, expression, connection):
decrypted_value = super().from_db_value(value, expression, connection)
if decrypted_value is not None:
return json.loads(decrypted_value)
return decrypted_value
def to_python(self, value):
if isinstance(value, str):
return json.loads(value)
return value
def get_prep_value(self, value):
value = super().get_prep_value(value)
if value is not None:
return json.dumps(value)
return value
def deconstruct(self):
name, path, args, kwargs = super().deconstruct()
return name, "your_project.fields.JSONPGPSymmetricalKeyField", args, kwargs
Add this code snippet to a new file in your project, say your_project/fields.py, and import JSONPGPSymmetricalKeyField in your models to use it:
from django.db import models
from .fields import JSONPGPSymmetricalKeyField
class MyModel(models.Model):
data = JSONPGPSymmetricalKeyField(passphrase="your_passphrase")
Keep in mind that you'll lose the ability to perform direct database operations on the JSON data. All queries will need to be done at the application level, after loading and decrypting the data.
To add support for JSONField in the form of JSONPGPPublicKeyField and JSONPGPSymmetricKeyField, you can create two new custom fields that extend the functionality of JSONField and PGPTextField respectively.
Here's an example implementation:
from django.db import models
from django.contrib.postgres.fields import JSONField
from pgcrypto_fields import PGPTextField
class JSONPGPPublicKeyField(JSONField, PGPTextField):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.validators = []
def from_db_value(self, value, expression, connection):
if value is None:
return value
return super().from_db_value(value, expression, connection)
def to_python(self, value):
if value is None:
return value
return super().to_python(value)
def get_prep_value(self, value):
if value is None:
return value
return super().get_prep_value(value)
class JSONPGPSymmetricKeyField(JSONField, PGPTextField):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.validators = []
def from_db_value(self, value, expression, connection):
if value is None:
return value
return super().from_db_value(value, expression, connection)
def to_python(self, value):
if value is None:
return value
return super().to_python(value)
def get_prep_value(self, value):
if value is None:
return value
return super().get_prep_value(value)
These custom fields can then be used in your models like so:
from django.db import models
from pgcrypto_fields import EncryptedTextField
from .fields import JSONPGPPublicKeyField, JSONPGPSymmetricKeyField
class MyModel(models.Model):
public_key = JSONPGPPublicKeyField()
symmetric_key = JSONPGPSymmetricKeyField()
encrypted_field = EncryptedTextField(key='symmetric_key')
This example shows how you can use the new custom fields in conjunction with the EncryptedTextField from pgcrypto_fields to create an encrypted field that uses the symmetric_key field as the encryption key
To add support for JSONField in the form of JSONPGPPublicKeyField and JSONPGPSymmetricKeyField, you will first need to create the corresponding encrypted field classes, as well as their respective form fields and widgets.
Here's some sample code you can use as a base for your implementation:
import json
from django import forms
from django.conf import settings
from django.contrib.postgres.fields import JSONField
from django.core.exceptions import ValidationError
from django.db.models import Field
from django.utils.translation import gettext_lazy as _
from pgcrypto.fields import *
from .widgets import EncryptedJSONWidget
class JSONEncoder(json.JSONEncoder):
def default(self, obj):
if hasattr(obj, "toJson"):
return self.default(obj.toJson())
return json.JSONEncoder.default(self, obj)
class JSONPGPPublicKeyField(PGPPublicKeyField):
empty_strings_allowed = False
form_class = JSONField
def topython(self, value):
if value is None:
return value
try:
return json.loads(value)
except json.JSONDecodeError:
raise ValidationError(
("%s value must be valid JSON.") % self.verbose_name, code="invalid"
)
def get_prep_value(self, value):
value = super().get_prep_value(value)
return json.dumps(value, cls=JSONEncoder)
class JSONPGPSymmetricKeyField(PGPSymmetricKeyField):
empty_strings_allowed = False
form_class = JSONField
def topython(self, value):
if value is None:
return value
try:
return json.loads(value)
except json.JSONDecodeError:
raise ValidationError(
("%s value must be valid JSON.") % self.verbose_name, code="invalid"
)
def get_prep_value(self, value):
value = super().get_prep_value(value)
return json.dumps(value, cls=JSONEncoder)
JSONPGPPublicKeyField
and JSONPGPSymmetricKeyField
:class JSONEncryptedFormField(forms.JSONField):
widget = EncryptedJSONWidget
def prepare_value(self, value):
if isinstance(value, (dict, list)):
return json.dumps(value, cls=JSONEncoder)
return value
class JSONPGPPublicKeyField(...):
def formfield(self, kwargs):
# Sets the default form field for the model field
defaults = {"form_class": JSONEncryptedFormField}
defaults.update(kwargs)
return super().formfield(defaults)
class JSONPGPSymmetricKeyField(...):
def formfield(self, kwargs):
# Sets the default form field for the model field
defaults = {"form_class": JSONEncryptedFormField}
defaults.update(kwargs)
return super().formfield(defaults)
EncryptedJSONWidget
in the widgets.py file:from django.forms.widgets import Textarea
class EncryptedJSONWidget(Textarea):
def value_from_datadict(self, data, files, name):
value = data.get(name)
if not value:
return None
try:
return json.loads(value)
except json.JSONDecodeError:
return value
@some1ataplace PRs for a new JSON field are welcome.
We have some data in JSON-field (django.contrib.postgres.fields.JSONField), but this package does not support it. Could it be possible to add this functionality?