Fixes and enhancements to subresource handling

[aparajita]

• Provided a way to configure a field to use as the pk for subresources using the subresource_pk field in Meta, so that individual embedded documents can reliably be addressed in a persistent way.
• If synthesized subresource pk's are being used, they start from 1, not zero. Zero is an invalid ObjectId, and when trying to create subresources, validate() is done on the parent, which validates embedded documents as well. If it sees a synthesized id of zero, it fails.
• Added support for sparse representation of ReferenceFields in an EmbeddedListField. If the 'full' field attribute is EmbeddedListField.SPARSE ('sparse'), then instead of returning an array of resource_uri, it returns an array of objects with the item "resource_uri" that contains the resource_uri. This allows clients to deal with embedded lists in a uniform way, as an array of objects.
• When embedded documents are deleted, their fields are checked for FileField, and the files associated with those fields are deleted.
• Small typo fixes

Signed off by: Aparajita Fishman aparajita@aparajita.com

[aparajita]

Didn't have time to update the tests for this, sorry.

[mitar]

It cannot be merged automatically. Can you update to the latest from the repository?

And please, when you will have time, make some tests for this new features.

[mitar]

instead of returning an array of resource_uri, it returns an array of objects with the item "resource_uri" that contains the resource_uri.

Interesting. But why name this sparse? I don't understand.

This allows clients to deal with embedded lists in a uniform way, as an array of objects.

Hm, I am not sure about this. Because this is breaking how Tastypie represents API. I would like to see that a library which knows how to talk to Tastypie API, can do this without knowing which data storage is used behind. So this feature should probably go to Tastypie. Also, one of interesting features of Tastypie is also its schema output which in theory allows clients to autoconfigure themselves to the API. I am not sure how this now interacts with this "sparse" feature.

[mitar]

BTW, personally, I think it is better that you have string vs. object. So clients knows, if it is string, it has to load the resource. If it is object, it can use it. How does client determines what it got with this "sparse" enabled?

[mitar]

Provided a way to configure a field to use as the pk for subresources using the subresource_pk field in Meta, so that individual embedded documents can reliably be addressed in a persistent way.

So if I understand correctly, this means you could have ObjectIds to access list elements? I was thinking about this, but I was thinking about it in a way that we could provide both ways at the same time. Index access or ObjectIds access. So the client can decide and use both.

A question, how you populate ObjectIds in subresources? Embedded documents do not have them themselves. You generate them as a default value for the field? How you assure their uniqueness? Because for main documents, database populates them.

I would really like to see this in action, especially in a form of a test. :-)

[mitar]

If synthesized subresource pk's are being used, they start from 1, not zero.

But who synthesize those pks? I don't understand. Wouldn't it be that in the case of using ObjectIds you decide what values pks should have, and just do not store 0 there but valid ObjectIds. And in the case of index access, you can start with 0.

[aparajita]

why name this sparse?

Because it's an object (as opposed to a raw resource_uri string), but sparsely populated with only the object's resource_uri.

[aparajita]

Because this is breaking how Tastypie represents API. So this feature should probably go to Tastypie.

Probably. It isn't specific to tpme.

[aparajita]

How does client determines what it got with this "sparse" enabled?

The idea is that you turn sparse mode on, which you know returns only resource_uri. Then if you want the full object, you pass a query parameter for example to get the full representation.

I was asked to do this for Ratatosk (https://github.com/wireload/Ratatosk), which I am using in a project. Ratatosk understands sparse mode, but doesn't deal so easily with non-object representations.

[aparajita]

But who synthesize those pks?

You do when using indexed embedded documents, the way it is in the current master. You are synthesizing the pk based on the position in the object_list, not based on a field.

in the case of index access, you can start with 0.

It wasn't working before, but now it is working, I'm not sure why. So I reverted the index change to be zero-based like before.

[mitar]

I finally have some time again to improve this module. Thank you for improvements. I will go over them and inject them into current version.

For sparse objects, please push this upstream to Tastypie. I will be then willing to make sure it also works here.

[mitar]

Thanks for this pull request!