gintas / django-picklefield

A pickled object field for Django
MIT License
180 stars 47 forks source link

Default field value not working in Django 3 #47

Closed OskarPersson closed 4 years ago

OskarPersson commented 4 years ago

For some reason after this commit in Django 3 I can no longer use default values for PickledObjectField in MySQL, I get the following error:

django.db.utils.OperationalError: (1101, "BLOB, TEXT, GEOMETRY or JSON column 'my_field' can't have a default value")

The error only occurs in MySQL 8.0.13+ (as mentioned in the commit) but the thing is that defaults has worked for me in Django 1.11-2.2 with MySQL 5.6-8.0.19 so I'm not sure why it has been constrained in Django 3.

Has anyone else had any experience with this or know a way around it?

(Maybe this is an issue more suited for Django itself but I thought I would start here since this is what I'm using for the fields that fails)

charettes commented 4 years ago

Hmm that looks like a Django specific issue, can you reproduce with a normal TextField as well given the default value you provide turns the pickled object into a str?

The logic seems to branch off db_type which relies on get_internal_type

https://github.com/django/django/blob/50cf183d219face91822c75fa0a15fe2fe3cb32d/django/db/models/fields/__init__.py#L665-L689

The thing is that this method returns the exact same thing for TextField and PickledObjectField.

https://github.com/django/django/blob/50cf183d219face91822c75fa0a15fe2fe3cb32d/django/db/models/fields/__init__.py#L2044-L2045

https://github.com/gintas/django-picklefield/blob/65afd8893dba9d4cdbeeabe5bd93db842be0c866/picklefield/fields.py#L205-L206

flokain commented 4 years ago

i can confirm i have the same issue when trying to pickle a Query object. it seems like the default values is not being serialized:

DB is postgres 10.1 django 3.0.3 python 3.7.4

eligible_team_query = PickledObjectField(default=Team.objects.all().query)

  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.SyntaxError: subquery must return only one column
LINE 1: ...query", "eligible_team_query") VALUES ('u17', '', (SELECT "p...
charettes commented 4 years ago

@flokain I think you're hitting a different issue that is due to a Django change and unrelated to the one reported by @OskarPersson.

Can you confirm that pickle.dumps(Team.objects.all().query) causes a similar crash?

Nice lefty backend btw :)

flokain commented 4 years ago

@charettes haha how small the world is!

i tested to save an object to the database and ran in to the same error. So I agree that this is a different issue.

I was able to pickle the query save it to the DB in a textfield and retrieve it correctly. Since this is working, i would assume it is an issue for django-picklefield, correct?

charettes commented 4 years ago

Closing as it's not clear whether Django or this package is at fault. From Django's storage perspective this field should be just like an other TextField.