SelfPublishModel overrides select_related (or just selects foreign keys we don't need)

[jdreben]

Hi --

We're using Swampdragon and self publishing models for a project that we are now optimizing. We've discovered that when we use self publishing models, iterating over them in templates causes swampdragon to send additional SQL queries to auth_user, and we can't fix this by using select_related on our self publishing model.

We're thinking that this is a bug in Swampdragon -- any ideas?

Thanks!

[hagsteel]

Could you either share a test that demonstrates this happening or alternatively share your model, serializer and router.

It's impossible to know if this is an SD issue or not. SD does not have anything to do with auth_user by default.

[jdreben]

We'll make you a minimal example asap, but we are using swampdragon auth as well.

[hagsteel]

Which version of SD are you using?

I can't imagine how the self pub model would do anything specific in a template. I'm intrigued to see how this is setup.

Are you talking about an actual Django template with a {% for x in object_list %} loop?

[jdreben]

Yes, we are, and the moment we remove the SelfPublishModel declaration from the model we lose almost 300 queries in loading the page. We use swampdragon 0.3.9 at the moment. We also use swampdragon-auth.

-- Router --

class BoardRouter(ModelRouter):
    route_name = 'board'
    serializer_class = BoardSerializer
    model = Board

    def get_object(self, **kwargs):
        return self.model.objects.get(pk=kwargs['id'], user=self.connection.user)

    def get_query_set(self, **kwargs):
        return self.model.objects.filter(user = self.connection.user)

-- Serializer --

class BoardSerializer(ModelSerializer):
    class Meta:
        model = 'userdata.Board'
        publish_fields = ('id', 'name', 'created', 'updated', 'user')
        update_fields = ('name',)

-- Models --

class BoardOrder(models.Model):
    order = models.IntegerField(null=True, blank=True)
    user = models.ForeignKey(User, related_name='user_order')
    board = models.ForeignKey(Board, related_name='board_order')
    subscribed = models.BooleanField(default = True)

    class Meta:
        ordering = ["order"]

class Board(SelfPublishModel, models.Model):
    serializer_class = BoardSerializer

    name = models.CharField(max_length=255, help_text="Name given by the user of the board.")
    user = models.ForeignKey(User, related_name='boards')

    created = models.DateTimeField(auto_now_add=True)
    updated = models.DateTimeField(auto_now=True)

    authorized_users = models.ManyToManyField(User)

    def __unicode__(self):
        return self.name

-- Views --

userboards = BoardOrder.objects.filter(user=request.user).select_related('board', 'board__user')

-- Template --

{% for bo in userboards %}
    ...
    {% if bo.board.user != request.user %}
    ...
    {% endif %}
{% endfor %}

For every iteration of the "for bo in userboards" loop, a SQL command is issued like

SELECT "auth_user"."id", "auth_user"."password", "auth_user"."last_login", "auth_user"."is_superuser", "auth_user"."username", "auth_user"."first_name", "auth_user"."last_name", "auth_user"."email", "auth_user"."is_staff", "auth_user"."is_active", "auth_user"."date_joined" FROM "auth_user" WHERE "auth_user"."id" = 352 LIMIT 21

If we comment out SelfPublishModel, all these queries disappear. Any ideas? We think it's a SD problem because removing SD from the model fixes it.

[hagsteel]

Ah this might be pre certain optimisations.

Could you try with 0.4.3: https://github.com/jonashagstedt/swampdragon/tree/0.4.3 Install it from github as it's not yet on PyPI. I suspect this will make the excessive queries go away