Backbone-relational with django-tastypie and non-full relations

[ulmus]

I'm using backbone-relational together with django-tastypie (as you seem to do yourself by the look of your backbone-tastypie project). I seem to get an error when using tastypie relations that are not defined as full, ie sent as resource Uri strings only. The related models in backbone-relational do not get instantiated. When sending full JSON representation of my related objects, they get instantiated ok.

I use this line from your backbone-tastypie plugin:

Backbone.Model.prototype.idAttribute = "resourceUri" // I have camelCased the JSON in tastypie

I have set up my model as follows

UserProfileModel = Backbone.RelationalModel.extend({});

UserModel = Backbone.RelationalModel.extend({
    relations: [
        {
            type: "HasOne",
            key: "profile",
            relatedModel: "UserProfileModel"
        }
    ]
});

I then do this:

var user = new UserModel({resourceUri: currentUserResourceUri});
user.fetch({
    success:function(){
        var userProfile = user.get("profile");
        userProfile.fetch() // <= this is where it debugs, as user.attributes.profile is undefined 
    }
});

When I send the following JSON from the server on user.fetch(), it works:

{
    resourceUri: "/api/v1/user/1/",
    firstName: "Jens",
    lastName: "Alm",
    profile: {
        resourceUri: "/api/v1/profile/1/",
        title: "Admin"
    }
}

When I send the following it fails, though, from how I understand it, it should work:

{
    resourceUri: "/api/v1/user/1/",
    firstName: "Jens",
    lastName: "Alm",
    profile: "/api/v1/profile/1/"
}

Am I misunderstanding how backbone-relational works? Shouldn't this also create a model that I can later fetch to fill with actual attributes?

[PaulUithol]

Well, this is a tricky issue.

If you simply pass in a string (or a list of strings, for a HasMany relation) as an attribute value, Relational doesn't automatically create models for each of them. If it did, the following would cause major issues, without much to be done about it:

var user1 = new UserModel({
    resourceUri: "/api/v1/user/1/",
    firstName: "Jens",
    lastName: "Alm",
    profile: "/api/v1/profile/1/"
});

var userProfile = new UserProfileModel({
    resourceUri: "/api/v1/profile/1/",
    title: "Admin"
});

This would cause a UserProfileModel with id /api/v1/profile/1/ to be created as soon as user1 is set up; then, another model with that same id is created when you create userProfile shortly after. This causes problems since:

• we can end up with a lot of (empty) dummy models if either fetching fails, data isn't available for some models, or for any other reason. I'd consider being allowed to iterate or use such a relation a bug, since other code will probably depend on fields to be available.
• we'll end up with multiple models with the same id (not least of all in Backbone.Store's collections), which is a problem.

So instead, I've chosen to have the Relation take the data out of it's associated attribute, store the contents in Relation.keyContents, then wait for a proper model with this id to be initialized, and set this one as the related model. The keyContents are not exposed though; ergo, user.get("profile"); yields null.

The options that I see for addressing this:

• fetch more data from the server; use full=True, or also fetch a list of UserProfileModels
• create models with just ids anyway (maybe only after enabling an option); I'm afraid this will do more harm than good though.
• expose the original attribute's content in some way:
  • have get return the attribute keys for missing related models; this would mean inconsistent return values though, so probably not a good idea.
  • introduce a separate method to get the original attribute data (something like getKey on Backbone.RelationalModel?).

[ulmus]

I understand. I don't want to send full data on all relations, as some of them would be quite large (also a matter of security, I don't want all profiles to be sent verbatim to the client unless the user is authorized to 'get' them).

What I could do of course is only use full=True relations as you suggest and then for those cases where I want the more "loose" binding, I define it manually in the code and not through relations. Another version is to write a Serializer in tastypie that gives the full=False relations as {resource_uri: "/api/v1/profile/1/"} instead of "/api/v1/profile/1/", it's kind of a hack, but it would do what I intended, even though you recommend against it.

For me, I'll probably go with the manual relations version, but I do think that some kind of option on the relation, such as createSkeletonModels or equivalent would be good with eg tastypie. Sending resource URI as relation is a common pattern for relational JSON. I'll see if I can get a pull request with some code for you to look at as a suggestion.

[PaulUithol]

Wouldn't the third option (exposing the keys, if no relation can be set up yet b/c of missing models/data) be something that would work for you? Looks to me like that would give you the most robust code in the end, since actual models and the 'skeletons' are logically separated, so you won't accidentally try to use a 'skeleton' model (because of forgetting a fetch call, a request timing out or throwing an exception, etc.). You could do something like this then:

var profile = user.get("profile");
if ( !profile && ( profileId = user.getKey('profile') ) ) {
    // No 'profile' there yet, but we do have an id. Fetch the profile using 'profileId';
    // the relation will be populated as soon as the data has arrived.
}

Maybe the user.get("profile") call could even trigger the fetching of the profile (lazy loading) in some way; although the return type for the get method would then probably be an XHR/deferred.. which is not something you'd want to check for everywhere.

[ulmus]

It would work for me. Also, I could do a Backbone.Model.fillRelations() method to fill all unfilled relations and fetch their results. With some jQuery deferred magic that method could return a deferred object that "executes" when all the results are back in.

I would still have to remember what attributes were sent as Full and not, but a simple _.isString() would solve that.

[PaulUithol]

Yeah, sounds good. I was thinking about something similar - only ithen named Backbone.RelationalModel.loadRelations(<key>) or fetchRelations ;). I could see how this would work for HasOne relations, but I'm a bit worried about HasMany.

I wouldn't want to fire of 10+ separate requests at the same time. Some frameworks (like Tastypie) offer a mechanism to retrieve a set of models (for example, /api/v1/<resource>/set/1;2;6;7/), but I don't know for other frameworks. Regardless, there should be a method to get such an url, so it can by used by load/fill/fetchRelations. Maybe as an optional second argument to, or as an optional method on Backbone.Collection that would be called from within loadRelations. If we can't determine such a url, individual requests will be made. What do you think?

[ulmus]

A Backbone.RelationalModel.fetchRelations(<key1>, <key2>, <key3>, ...) with keys as optional argument and returning a deferred object that evaluates when all keys have returned would be very clean and good. Performance might be an issue with collections as you mention, but I would probably try to keep it clean anyway, perhaps with a loadRelations as you suggest that per default fires a lot of ajax requests but that could be overridden and tweaked for specific backends if necessary. If you want performance, I'd suggest using full=True or equivalent, just document it properly.

I´d love to make a pull request, but I'm not quite familiar with your code, so I'll probably leave this to you if you get around to it and think it's a good addition, otherwise leave it and I'll look into it in a couple of weeks when my current project winds down a bit.

[ulmus]

BTW, it's a great plugin you've made! I tried to make my own automatic relationship builder, but after a week of frustrating, hard to track down bugs, I conceded :). Backbone-relational works splendidly for my needs and looks very well designed!

[ulmus]

FYI, I moved on to django-rest-framework which, I think, is much cleaner implemented than tastypie and easier to understand and customize. This means that I can specify how to send related attributes on the server side making it easy to me to send one set of attributes when sending the model through a relation and then complete the model on a fetch as per your first suggestion.

Theoretically I see the problem your describing with this approach, that I don't know whether the model i fully fetched or not. In practice, these are corner-cases for me, so there are few enough of them that I can keep track of it in code.

As a general solution, the suggestion we discussed with getKey() and assoicated methods still seems like a good idea, though I might end up not actually using it.

[PaulUithol]

Thanks for the tip, I'll mention it around here. We also have to override just about everything in tastypie (and then, still..).

I'll try to get something working for this issue pretty soon, let's see then.

[PaulUithol]

Alright, how about this? fetchRelated(<key>, <options>) will make separate requests by default, or will use the related collection's url method to obtain a single url for a set of models if possible. Which works pretty nicely in combination with being able to specify the collectionType ;).

[ulmus]

Seems very good! Being able to specify server specific capabilities seems like something that should be added in Backbone.js, but for now the function sniffing approach should work! A noet about it in documentation though, I could see scenarios where you define an url()-method without being able to handle sets of ids.

Perhaps solve it as Backbone does with emulateHTTP, add an attribute to the Backbone class where the developer can specify that the server supports fetching multiple Ids (fetchMultipleIds?) Seems like it should be more of a generic setting, a serverCapabilities bitmap perhaps, but that's out of scope for this of course!

[PaulUithol]

Yep, I'll update the docs shortly. Regarding the url method, for now I've solved it by comparing the url we get for a set of models with the (default) url returned by the collection when url is called without arguments. If they're the same, apparently we can't request a set of models. See line 918 and the comment before it:

// An assumption is that when 'Backbone.Collection.url' is a function, it can handle building of set urls.
// To make sure it can, test if the url we got by supplying a list of models to fetch is different from
// the one supplied for the default fetch action (without args to 'url').
if ( setUrl && setUrl !== rel.related.url() ) {
    // request the set

[PaulUithol]

Alright, docs updated. Liked your 'zoo' example, so I've used that as a first short example.