Failing on nested list of embedded documents

[leo-naeka]

Hi, First of all, thanks thanks to every contributor on this project, it is very useful when playing with tastypie !

I am encountering a little problem with nested list of embedded documents. This is a little diff on the test project to reproduce the issue :

--- a/tests/test_project/test_app/documents.py
+++ b/tests/test_project/test_app/documents.py
@@ -14,6 +14,7 @@ class StrangePerson(Person):
 class EmbeddedPerson(mongoengine.EmbeddedDocument):
     name = mongoengine.StringField(max_length=200, required=True)
     optional = mongoengine.StringField(max_length=200, required=False)
+    coworkers = mongoengine.ListField(mongoengine.EmbeddedDocumentField("EmbeddedPerson"))

 class EmbeddedStrangePerson(EmbeddedPerson):
     strange = mongoengine.StringField(max_length=100, required=True)

--- a/tests/test_project/test_app/api/resources.py
+++ b/tests/test_project/test_app/api/resources.py
@@ -38,6 +38,8 @@ class EmbeddedStrangePersonResource(resources.MongoEngineResource):
         object_class = documents.EmbeddedStrangePerson

 class EmbeddedPersonResource(resources.MongoEngineResource):
+    coworkers = fields.EmbeddedListField(of='test_project.test_app.api.resources.EmbeddedPersonResource',     attribute='coworkers', full=True, null=True
+    
     class Meta:
         object_class = documents.EmbeddedPerson
         allowed_methods = ('get', 'post', 'put', 'patch', 'delete')

After populating the database :

$ ./manage.py shell
Python 2.7.3 (default, Apr 20 2012, 22:39:59) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from test_project.test_app.documents import *
>>> leo = EmbeddedPerson(name = 'leo')
>>> tom = EmbeddedPerson(name = 'tom')
>>> max = EmbeddedPerson(name = 'max')
>>> leo.coworkers = [tom,max]
>>> tom.coworkers = [max]
>>> liste = EmbeddedListFieldTest(embeddedlist = [leo,tom,max])
>>> liste.save()
<EmbeddedListFieldTest: EmbeddedListFieldTest object>

And when accessing to /api/v1/embeddedlistfieldtest/, we get an error : "init() takes at least 3 arguments (2 given)" :

Traceback (most recent call last):

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/local/lib/python2.7/site-packages/tastypie/resources.py", line 192, in wrapper
    response = callback(request, *args, **kwargs)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/local/lib/python2.7/site-packages/tastypie/resources.py", line 397, in dispatch_list
    return self.dispatch('list', request, **kwargs)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/tastypie_mongoengine/resources.py", line 251, in dispatch
    return super(MongoEngineResource, self).dispatch(request_type, request, **kwargs)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/local/lib/python2.7/site-packages/tastypie/resources.py", line 427, in dispatch
    response = method(request, **kwargs)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/local/lib/python2.7/site-packages/tastypie/resources.py", line 1037, in get_list
    to_be_serialized['objects'] = [self.full_dehydrate(bundle) for bundle in bundles]

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/tastypie_mongoengine/resources.py", line 292, in full_dehydrate
    return super(MongoEngineResource, self).full_dehydrate(bundle)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/local/lib/python2.7/site-packages/tastypie/resources.py", line 654, in full_dehydrate
    bundle.data[field_name] = field_object.dehydrate(bundle)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/tastypie_mongoengine/fields.py", line 173, in dehydrate
    m2m_dehydrated.append(self.dehydrate_related(m2m_bundle, m2m_resource))

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/local/lib/python2.7/site-packages/tastypie/fields.py", line 514, in dehydrate_related
    return related_resource.full_dehydrate(bundle)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/tastypie_mongoengine/resources.py", line 296, in full_dehydrate
    return super(MongoEngineResource, self).full_dehydrate(bundle)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/local/lib/python2.7/site-packages/tastypie/resources.py", line 654, in full_dehydrate
    bundle.data[field_name] = field_object.dehydrate(bundle)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/tastypie_mongoengine/fields.py", line 170, in dehydrate
    m2m_resource = self.get_related_resource(m2m)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/local/lib/python2.7/site-packages/tastypie/fields.py", line 462, in get_related_resource
    related_resource = self.to_class()

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/tastypie_mongoengine/fields.py", line 188, in <lambda>
    self._to_class_with_listresource = lambda api_name=None: base_with_listresource(self._resource, self.instance_name, api_name)

  File "/home/leo/Coding/django-tastypie-mongoengine/tests/tastypie_mongoengine/resources.py", line 554, in __init__
    self.parent = parent(api_name)

TypeError: __init__() takes at least 3 arguments (2 given)

And then, to be honest, I don't know what's happening : as far as I know, parent should be a MongoEngineResource, inherited from tastypie.ModelResource, inherited from tastypie.Resource, which only takes the api_name as parameter...

Thanks in advance for spending your time on this. Leo.

[mitar]

Can you please prepare a test case with this problem? And make a pull request? Test case should populate data through RESTful API, not directly in the database.

[mitar]

Uh, nested lists? This was never meant to be supported. This is a feature request.

Are you sure it is needed? For me it looks a bit strange to have a board with list of posts. So everytime you will want to read one post you will load the whole board? In nonrel databases you should optimize to the most common use. Probably the most common use is to display posts. So post should be a document. And board could just be a field to limit the posts to.

Or was this just an example for test?

[leo-naeka]

Yes, of course, that was just for testing, to be clear with a common example, and... I wasn't so much inspired !

In my case, I have a complex object with a part of its data that should be versionned in time and I use for that a list of "revisions" between the object and the versionned data. That data is never manipulated without manipulating the object.

This is also a great advantage with nonrel database to avoid complex (and resource-intensive) relationships with recursive embedded objects.

[mitar]

OK. But you probably don't do versioning through REST but it is done automatically in the background and you do not expose those versions through API? Or you do?

So I will try to fix that you can define such resources. But to be able to access them through URI directly, this will be for some other time (and of course patches are welcome).

[leo-naeka]

Yes, versionning is done in the background, but I would have the revisions list exposed so that a user could, for example, view/compare and revert to a specific revision.

Having directly access to a revision through the API is not required since each of them are listed in the main object. But only giving the current revision and allowing to get others on demand would be indeed (no waste of bandwidth, small objects).

Thank you for spending time on this issue, I will try for sure to have this working and submit you some patches.

[mitar]

Fixed. Please check.

[leo-naeka]

Great ! Working wery well ! Thanks a lot.

I will probably try to get access to the subresources through the API in the next couple of weeks. I'll keep you informed and eventually submit a pull request.