Dict keys are not unique in the string in the database

[hmpf]

I use jsonfield to store a blob of the following format (newlines added for readability):

{
    "213":  {
        "stringfield": "bla",
        "datafield": {"max": 3, "min": 1}
    },
    "432": {
        "stringfield": "",
        "datafield": [1,2,3]
    }
}

I change one entry at a time:

new = {"stringfield": "", "datafield": [1,2]}
model.data["432"] = new
model.save()

Sometimes, the string stored in the database is not a serialization of a dict, because the keys are not unique:

{
    "213":  {
        "stringfield": "bla",
        "datafield": {"max": 3, "min": 1}
    },
    "432": {
        "stringfield": "",
        "datafield": [1,2,3]
    },
    "432": {
        "stringfield": "",
        "datafield": [1,2]
    }
}

I have SQL logging turned on, and what I see is that Django sends

    "432": {
        "stringfield": "",
        "datafield": [1,2,3]
    },
    "432": {
        "stringfield": "",
        "datafield": [1,2]
    }

.. and that's what the server stores. Then on select, python eventually converts to dict, and chooses always the final entry, which is the first version of the stored entry. The admin shows a single entry for 432 for instance. The end result is that it seems 432 haven't changed, because the new data can never be read back.

It looks to me like jsonfield is doing something funky. This is jsonfield version 2.0.2.

[hmpf]

.. I'm stuck on postgres 9.3 btw.

[rpkilby]

Hi @hmpf. I tested this locally but was unable to recreate the issue. It's possible this was fixed in the 3.0 rewrite. Here's the sample test case I used:

class ExampleTest(TestCase):
    @override_settings(DEBUG=True)
    def test_ex(self):
        from django.db import connection
        instance = JSONNotRequiredModel.objects.create(json={
            "213":  {
                "stringfield": "bla",
                "datafield": {"max": 3, "min": 1},
            },
            "432": {
                "stringfield": "",
                "datafield": [1, 2, 3],
            }
        })

        instance.json["432"] = {"stringfield": "", "datafield": [1, 2]},
        instance.save()

        for q in connection.queries:
            print('*' * 80)
            print(q['sql'])

Which outputs:

********************************************************************************
INSERT INTO "tests_jsonnotrequiredmodel" ("json") VALUES ('{"213": {"stringfield": "bla", "datafield": {"max": 3, "min": 1}}, "432": {"stringfield": "", "datafield": [1, 2, 3]}}')
********************************************************************************
UPDATE "tests_jsonnotrequiredmodel" SET "json" = '{"213": {"stringfield": "bla", "datafield": {"max": 3, "min": 1}}, "432": [{"stringfield": "", "datafield": [1, 2]}]}' WHERE "tests_jsonnotrequiredmodel"."id" = 1

Please let me know if this is still an issue for you.