doableware / djongo

Django and MongoDB database connector
https://www.djongomapper.com
GNU Affero General Public License v3.0
1.89k stars 355 forks source link

using Listfield with DRF get value as None #183

Open ahmedakef opened 6 years ago

ahmedakef commented 6 years ago

One line description of the issue

i send coordinates as list of two float numbers but Django rest frame word reports this :

f'Value: {value} must be of type list'
ValueError: Value: None must be of type list

Python script

my model is :

class GeoJSON(models.Model):
    type        = models.CharField(max_length=15, default = "Point")
    coordinates = models.ListField()

    class Meta:
        abstract = True

Traceback


Traceback (most recent call last):
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/core/handlers/base.py", line 124, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/rest_framework/viewsets.py", line 103, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/rest_framework/views.py", line 483, in dispatch
    response = self.handle_exception(exc)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/rest_framework/views.py", line 443, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/rest_framework/views.py", line 480, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/rest_framework/mixins.py", line 21, in create
    self.perform_create(serializer)
  File "/home/ahmed/developement/call_for_code/disaster-management/backend/disasters_management/disasters/views.py", line 30, in perform_create
    serializer.save(owner=self.request.user)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/rest_framework/serializers.py", line 214, in save
    self.instance = self.create(validated_data)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/rest_framework/serializers.py", line 940, in create
    instance = ModelClass.objects.create(**validated_data)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/query.py", line 413, in create
    obj.save(force_insert=True, using=self.db)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/base.py", line 718, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/base.py", line 748, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/base.py", line 831, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/base.py", line 869, in _do_insert
    using=using, raw=raw)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/query.py", line 1136, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1288, in execute_sql
    for sql, params in self.as_sql():
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1241, in as_sql
    for obj in self.query.objs
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1241, in <listcomp>
    for obj in self.query.objs
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1240, in <listcomp>
    [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1182, in prepare_value
    value = field.get_db_prep_save(value, connection=self.connection)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/django/db/models/fields/__init__.py", line 790, in get_db_prep_save
    return self.get_db_prep_value(value, connection=connection, prepared=False)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/djongo/models/fields.py", line 497, in get_db_prep_value
    mdl_ob[fld.attname] = fld.get_db_prep_value(fld_value, connection, prepared)
  File "/home/ahmed/pyenv_16/disaster/lib/python3.6/site-packages/djongo/models/fields.py", line 98, in get_db_prep_value
    f'Value: {value} must be of type list'
ValueError: Value: None must be of type list

my JSON data is :

{
    "type"  :   "Point",
    "coordinates"   : [5,8]
}
lhmisho commented 5 years ago

It's worked for me ...

numbers = models.ListField(blank=True, null=True, default=[ ])
txusdumort commented 5 years ago

hi.

I've a similar problem, wen the context is described below.

I'm using MongoDB the connection is provided by Djongo, over is being used DRF to manage all request to mi API.

My data (profile) is structured like next Json object

{
    "name" : "profile name",
    "description" : "this is a description",
    "params" : "X1, X2,X3, etc",
    "config" : "CONFIG OF DEVICE",
    "user" : {
        "name" : "user name",
        "middle_name" : "test middle name",
        "last_name" : "test last name",
        "email" : "test@test.com",
        "institute" : {
            "name" : "MIT",
            "place" : {
                "coordinates" : [ 30.0, 101.0,  0.0 ],
                "type" : "Point"
            },
            "country" : "US"
        }
    },
    "place" : {
        "coordinates" : [  90.0, 901.0, 10.0 ],
        "type" : "Point"
    },
    "devices" : [ 
        {
            "name" : "DEVICE 1",
            "verification_code" : "",
            "verificated" : 0,
            "configuration" : "kjk",
            "places" : [ 
                {
                    "coordinates" : [ 30.0,  101.0,  0.0 ],
                    "type" : "Point"
                }, 
                {
                    "coordinates" : [  31.0, 102.0, 1.0 ],
                    "type" : "Point"
                }
            ]
        }
    ]
}

I know, the coordinates are wrong, but is just for test.

Well I send that object to my view and then to the ProfileSerializer, this get the responsible to check the embedded objects (each one have your own serializer). After checking data, the info is saved without problem as you can see in next picture:

Screenshot 2019-03-21 at 14 44 05

But the problem is when I try to. retrieve all profiles. Just the coordinates are null, Other embedded objects are retrieved in good way, only the Place Object is malformed. Next, I'll show you the response:

[
{
        "id": 22,
        "name": "profile name",
        "description": "this is a description",
        "params": "X1, X2,X3, etc",
        "config": "CONFIG OF DEVICE",
        "user": {
            "name": "user name",
            "middle_name": "test middle name",
            "last_name": "test last name",
            "email": "test@test.com",
            "institute": {
                "name": "MIT",
                "place": {
                    "coordinates": **null**,
                    "type": "Point"
                },
                "country": "US",
                "created_at": "2019-03-21T20:43:33.928000Z"
            },
            "created_at": "2019-03-21T20:43:33.959000Z"
        },
        "place": {
            "coordinates": **null**,
            "type": "Point"
        },
        "devices": [
            {
                "name": "DEVICE 1",
                "verificated": 0,
                "configuration": "kjk",
                "places": [
                    {
                        "coordinates": **null**,
                        "type": "Point"
                    },
                    {
                        "coordinates": **null**,
                        "type": "Point"
                    }
                ],
                "created_at": "2019-03-21T20:43:33.898000Z"
            }
        ],
        "created_at": "2019-03-21T20:43:33.976000Z"
    }
]

For this questions only I'll describe/show the serializer of one object, but if you need some info I'll get you as soon as possible.

Models

class Place(models.Model):
    coordinates = models.ListField(blank=True, null=True, default=[0.0, 0.0, 0.0])
    type = models.CharField(max_length=10, default="Point")
    objects = models.DjongoManager()

class Profile(models.Model):
    name = models.CharField(max_length=200)
    description = models.TextField(default="Without Description")
    params = models.TextField(default="No params")
    config = models.CharField(max_length=200)
    user = models.EmbeddedModelField(
        model_container=User
    )
    place = models.EmbeddedModelField(
        model_container=Place
    )
    devices = models.ArrayModelField(
        model_container=Device
    )
    created_at = models.DateTimeField(auto_now_add=True)
    objects = models.DjongoManager()

Serializers

class PlaceSerializer(serializers.ModelSerializer):
    coordinates = serializers.ListSerializer(
        child=serializers.FloatField(),
    )
    class Meta:
        model = Place
        fields = ('id', 'coordinates', 'type')

class ProfileSerializer(serializers.ModelSerializer):
    user = UserSerializer( )
    place = PlaceSerializer()
    devices = DeviceSerializer( many=True)

    class Meta:
        model = Profile
        fields = ('id', 'name', 'description', 'params', 'config',
                  'user', 'place', 'devices', 'created_at')
        depth=8

    def create(self, validated_data):
        # get principal fields
        user_data = validated_data.pop('user')
        **place_data = validated_data.pop('place')**
        devices_data = validated_data.pop('devices')

        # get nested fields

        # devices nested fields
        devices = []
        for device in devices_data:
            places = []
            places_data = device.pop('places')
            for place in places_data:
                places.append( **Place(coordinates=place['coordinates'], type=place['type'])** )

            device['places'] = places
            devices.append( Device.objects.create(**device) )
        validated_data['devices'] = devices
        # user nested fields
        institute_data = user_data.pop('institute')
        place = institute_data.pop('place')
        institute_data['place'] = Place(coordinates=place['coordinates'], type=place['type'])
        user_data['institute'] = Institute.objects.create(**institute_data)
        validated_data['user'] = User.objects.create(**user_data)
        profile = Profile.objects.create(**validated_data)
        return profile

I've defined PlaceSerializer on many ways but all of them gets the same result, Below describe this ways

CASE 1

class PlaceSerializer(serializers.ModelSerializer):
    coordinates = serializers.ListSerializer(
        child=serializers.FloatField(),
    )

CASE 2

class CoordinatesSerializer(serializers.ListSerializer):
        child=serializers.FloatField()

class PlaceSerializer(serializers.ModelSerializer):
    coordinates = CoordinatesSerializer()

CASE 3

class PlaceSerializer(serializers.ModelSerializer):
    coordinates = serializers.ListField(
        child=serializers.FloatField()
    )

CASE 4

class PlaceSerializer(serializers.ModelSerializer):
    coordinates = serializers.ListField()

CASE 5

class PlaceSerializer(serializers.ModelSerializer):
    coordinates = serializers.ListSerializer()
#gives error for child is not present

I had changed the types, CharField, IntegerField, FloatField, etc with same results.

Another tests that I've done are append to serializer the methods create, update, to_representation, to_internal_value, all of this for to manage in a better way the info that will saved o retrieved but any works. Another curiosity, if I add a simple Listfield like [10,90,1], is saved and retrieved without problem in contrast when this ListField is inside Place Objects

Please if you know how to solve this I'll glad to you.