Support for filtering on embedded list field subresources

[mitar]

Support for filtering on embedded list field subresources.

[eelkeh]

I need support for filtering EmbeddedListField and ReferencedListField, any pointers on how to implement these would be greatly appreciated!

[mitar]

Probably you just have to extend this code with implementation for different filters.

[m-vdb]

Hi @mitar I need this feature, and I'm looking at tastypie and tastypie_mongoengine code. I have some questions.

First, in order to stay with mongoengine __ way of filtering, we need to filter EmbeddedDocumentField, EmbeddedListField and DictField this way. However, I found that tastypie uses the is_related attribute to authorize such filtering, and this attribute is set to False in all those fields. We could :

• either put the flag to True even if it's not really relevant, and benefit from already coded methods
• use another flag and program everything for tastypie_mongoengine

Then, we might have some problems with EmbeddedDocument that contain other document or dict fields. I don't really know how to handle those cases.

Do you have any idea ?

Thanks :)

[mitar]

So there is a difference between subresources and subdocuments. Subdocuments, this is still filtering on the top-level resource and should work by passing through filters to MongoEngine.

For subresources (/api/v1/something/<id1>/field/<id2>/) you have to extend this code because we are using our own list for traversing lists embedded in the resource. Now that I am thinking, you might be able to map those filters on subresources to top-level filter.

I currently do not have time to implement this. But pull requests are welcome.

[m-vdb]

In fact, I'm focusing on subdocuments. Here is my mongoengine document :

class Page(mongoengine.EmbeddedDocument):
    parameters = mongoengine.DictField()

class Client(mongoengine.Document):
    pages = mongoengine.ListField(mongoengine.EmbeddedDocumentField(Page))

And I have resources that are properly defined on those. What I would like to perform is :

GET localhost:8000/api/v1/client/?format=json&pages__parameters__key=value

How do you think this is gonna work ? Here I need :

• ability to filter on Lists of EmbeddedDocuments
• ability to filter key/value from a DictField

thanks @mitar !

[mitar]

Yes, this should be possible by passing those filters through. But some customization for Mongoengine backend will be needed. See how it is implemented for Django ORM.

[m-vdb]

Ok I managed to do it by overriding the build_filters, but this is not cool. The list filtering works

pages__name=my_name

but the DictField filtering doesn't. I see no one that tried to perform this (stackoverflow, github, etc...) but this looks very pythonic for me. So maybe I do something wrong. Do you think we need to tweak the DictField tastypie's class ?

[mitar]

Ok I managed to do it by overriding the build_filters, but this is not cool.

Why it is not? build_filters is from Django ORM and you have to make support for MongoEngine somwhere.

About DictField, does filtering on it work in pure MongoEngine? So run in Django shell filtering on DictFields in MongoEngine and see if it works. I think Tastypie should just ignore fields and simply pass everything to MongoEngine. There might be some security issues though. And maybe you do not want to allow filtering on everything. But for now try to support passing everything.

[m-vdb]

It's not cool because it's custom for now, I didn't implement anything generic.

Yes you can filter mongo dict with dot ., and this is translated to __ at mongoengine level. Plus, I sense that it would also make sense for tastypie (not only tastypie_mongoengine) to filter onto DictField. So basically, what we could have is:

• another constant similar to ALL_WITH_RELATIONS but for dicts (that would restrict filtering onto the dict we want, for security reasons as you said)

• the ability to filter dictionaries like this:

  resource_value = {"a_dict": {"param": "value"}} GET /api/v1/resource/?format=json&a_dict__param=value

Does that make sense ? If so, I could start implementing it. But I don't know where to start (huge files in tastypie...), any hint for me ?

Thanks for your help :koala:

[mitar]

I am not sure Tastypie has any notion about what exactly is the field of. You should just pass filters to MongoEngine and this is it.

I think build_filters is exactly what you need to modify.