ArrayReferenceField with DRF

[iserranoe]

I am using arrayreferencefield with Django Rest Framework to post the documents. I managed not to have any errors posting but the arrayreferencefield appears empty. I don't want to use the arrayreferencefield to fill the Child document, but just to reference, that why I defined Datos_online =CodeSerializer(many=True, read_only=True)

My simplified code is as follows:

models.py

from djongo import models

class Code(models.Model):
    codigo = models.CharField(max_length=200,primary_key=True, default=None, editable=True)
    unidades = models.CharField(max_length=10, blank=True)
    objects = models.DjongoManager()
    class Meta:
        verbose_name_plural = "3. Datos"

class KPI(models.Model):
    id_kpi = models.CharField(max_length=200, primary_key=True)
    Datos_online = models.ArrayReferenceField(
        to = Code, 
        on_delete = models.CASCADE,
        blank=True,
        null=True,
        related_name="online"
    )
    objects = models.DjongoManager()
    class Meta:
        verbose_name_plural = "4. KPIs"

serializers.py

from rest_framework import serializers
from .models import Code, KPI
from djongo import models

class CodeSerializer(serializers.ModelSerializer):
    class Meta:
        model = Code
        fields = '__all__'

class KPISerializer(serializers.ModelSerializer):
    Datos_online =CodeSerializer(many=True, read_only=True)
    class Meta:
        model = KPI
        fields = '__all__'

The post code in bash is:

$curl -d '''{"id_kpi":"001", "Datos_online":["0011","0012"]}''' -H "Content-Type: application/json" -X POST http://localhost:8000/postmongo/kpi/
${"id_kpi":"001","Datos_online":[]}

I tried many of the solutions recommended in the post, as #115, but it didn't work.

Any ideas? I am a bit lost right now...

[SomeoneInParticular]

Hello @iserranoe

Sorry for the delay; I saw your problem a week ago and, between work and how unusual ArrayReferenceFields are implemented (inherits from foreign key, acts like many to many key, lacks through table, among other weirdness), I only recently figured it out.

From the Djongo Documentation:

"The ArrayReferenceField behaves exactly like the ManyToManyField"

As a result, you cannot directly set the value of the field via a serializer field's implicit setup; instead you have to create custom 'create' and 'update' functions which use the Many-to-Many manager functions to create/modify your model. (see Writable Nested Serializers and Relation Managers for more details on that).

For example:

...
class KPISerializer(serializers.ModelSerializer):
    Datos_online = PrimaryKeyRelatedField(
        many=True, 
        queryset=Code.objects.all(),
        blank=True  # Generally good practice for relations, also required for this setup
    )

    def create(self, validated_data):
        datos_vals = validated_data.pop('Datos_online', [])  # Extract relation data
        kpi_instance = KPI.objects.create(**validated_data)  # Save the instance w/o relations
        kpi_instance.Datos_online.add(*datos_vals)  # Add our relations to the instance
        kpi_instance.save()  # Save the instance w/ relations added
        return kpi_instance

    def update(self, validated_data, instance):
        # Update which can add new relations, but CANNOT remove them
        pk = validated_data.pop('pk', None)  # Fetch our target pk
        datos_vals = validated_data.pop('Datos_online', [])  # Extract relation data
        kpi_instance = KPI.objects.fetch(pk=pk).update(**validated_data)  # Update non-relational data
        kpi_instance.Datos_online.add(*datos_vals)  # Add our relations
        kpi_instance.save()  # Save our instance w/ relations added
        return kpi_instance

    class Meta:
        model = KPI
        fields = '__all__'
...

This means we have to make 2 queries (at least) to the database, and Djongo can't really do much about it. Doing so would either conflict with the Django ORM standard, or result in the field acting fundamentally different from that of Django's Many-to-Many field (which it is meant to replace).

Some other caveats as well:

• ArrayReferenceFields do not create through models in the database. As a result, unlike native Django Many-to-Many fields, you cannot explicitly serialize that table nor rely on it to bypass the relation manager.
• ArrayReferenceFields also do not create reverse relation fields in the related model's database table either; despite this, the reverse field still is still generated and acts as another relation manager. This function is currently broken, however, always returning an empty queryset (some debugging I did for this answer seems to show that the query string is written assuming the array of pks (stored in ArrayReferenceField) is actually a single pk, and thus 'no query matching exists'). Hopefully this will be patched soon!

Hope this helps!

[iserranoe]

Thank you @SomeoneInParticular , I tried to implement your solution but I got the following error:

KPI matching query does not exist

I don't know what I am doing wrong...; anyway, I am finally not using the post api to fill the data base but just the admin site and mongo shell, and the get API to obtain the data, which is working fine,

[SomeoneInParticular]

I'm glad part of it is working now at least; do you know where that error is being thrown? (Update, create, or elsewhere?)

[iserranoe]

I am afraid I don't know... As I understand, "create" should be made to create a new document, shouldn't be? But I don't want to create a new document, just reference it, I am a bit confused...

[SomeoneInParticular]

Firstly, I made a slight error in my code; it should be:

kpi_instance = KPI.objects.get(pk=pk).update(**validated_data)

not

kpi_instance = KPI.objects.fetch(pk=pk).update(**validated_data)

Also, not that the add function in both create and update takes model objects, not the primary keys of those object. These issues might be the source of your error; my extreme apologies for that flub. If that's all you need/want, you can stop reading here.

You are correct w/ regards to the create function, though; its there to create an instance of a document representing that model in the database. However, with the ArrayReferenceField, as implemented above, we include creating a list of pks, corresponding to existing documents already in the database. So, upon creation, what you are supplying is simply a list of the pk values, not the instances themselves. For example, using a python console (assuming the code mentioned prior is already in use/imported):

# Creating the code to be referenced
code1 = Code.objects.create(
    codigo='print("Hello World!")',
    unidades='UTF-8'
)
code2 = Code.objects.create(
    codigo='print("Goodbye!")',
    unidades='UTF-8'
)

# Assuming no other code has been inserted into the database before,
# this will be [1, 2], as code1 will have an ID of 1, and code2 will
# have an ID of 2. This also means code with other ID's do not exist,
# for the purpouses of this demonstration
code_list = [code1, code2]

# The following code is what REST runs in the background on data submission.
# For the purpouses of demonstration I'm just making it explicit
def rest_submit(data):
    serializer = KPISerializer(data=kpi_data) # Serializer instantiation
    if not serializer.validate():  #  Confirm that data is invalid
        print(serializer.errors)  # `serializer.errors` contains the error info
    serializer.save()  # Try to save data to database

# Valid creation (code w/ provided ids exist)
valid_kpi_data = {
    'id_kpi': '001',
    'Datos_online': code_list  # [code1, code2]
)

rest_submit(valid_kpi_data)  # Works! All code pk's correspond to documents in the database

# Invalid creation (no code3 exists in the database)
code3 = Code(codigo="print('ERROR')", unidades='UTF-8')  # Not saved to database

invalid_kpi_data = {
    'id_kpi' '002',
    'Datos_online': [code1, code3]
}

rest_submit(invalid_kpi_data) # Error; `Code` matching query does not exist

The same is true for update functions; just replace serializer = KPISerializer(data=kpi_data) with serializer = KPISerializer(instance=<a KPI instance>, data=kpi_data) within the rest_submit function; the logics is pretty much identical.