googleapis / python-spanner-django

Cloud Spanner database backend for Django
BSD 3-Clause "New" or "Revised" License
90 stars 28 forks source link

Issue with JsonField in ORM Causing Data Retrieval Failure #905

Open murabo opened 2 months ago

murabo commented 2 months ago

I am encountering an issue when using JsonField in a Django model with Google Spanner. After saving data into a table that contains a JsonField through the Django ORM, I am unable to retrieve the data due to a TypeError.

The error occurs when attempting to query the table, and the following traceback is returned:

from django.db import models
class JsonTest(models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    body = models.JSONField(null=True, blank=True)
    is_processed = models.BooleanField(default=False)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/query.py", line 256, in __repr__
    data = list(self[:REPR_OUTPUT_SIZE + 1])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/query.py", line 280, in __iter__
    self._fetch_all()
  File "/Users/murabo/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/query.py", line 1324, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/query.py", line 68, in __iter__
    for row in compiler.results_iter(results):
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1116, in apply_converters
    value = converter(value, expression, connection)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/fields/json.py", line 83, in from_db_value
    return json.loads(value, cls=self.decoder)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/.pyenv/versions/3.12.0/lib/python3.12/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not JsonObject

It seems like the JsonField data type is not being handled correctly when reading from the database, possibly related to how the data is being serialized and deserialized.

Steps to reproduce:

Create a model with a JsonField. Save data using the Django ORM. Attempt to query the table containing the JsonField. Expected behavior: The data should be retrieved and deserialized correctly.

Actual behavior: The query fails with a TypeError, indicating that the JSON object must be str, bytes, or bytearray, but JsonObject is being returned.

Any help on resolving this issue would be greatly appreciated!