django-mptt / django-mptt

Utilities for implementing a modified pre-order traversal tree in django.
https://django-mptt.readthedocs.io/
Other
2.88k stars 467 forks source link

Duplicated queries in Django Admin - List Display #811

Closed YDA93 closed 2 years ago

YDA93 commented 2 years ago

I need to retrieve and display all parents at once in admin list display.

Currently I added this function to admin class and is working perfectly:

def category(self, obj):
    family = (
        obj.parent.get_ancestors(ascending=False, include_self=True)
        .select_related("parent")
        .values_list("name", flat=True)
    )
    return " ➤ ".join(family) # Will display Parent A ➤ Parent B ➤ Parent C

However this came with many duplicated queries.

select_related have greatly reduce the queries but still many more are there.

How can I approach this? Perhaps there a way to prefetch_related instead?

Thank you,

Screen Shot 2022-07-04 at 9 14 16 PM

matthiask commented 2 years ago

You could try using .select_related("parent__parent__parent") or something but maybe this will perform worse. You'll have to try it out.

Maybe you could go a different route and use the draggable MPTT admin to visualize the tree structure?

YDA93 commented 2 years ago

@matthiask

.select_related("parent__parent__parent") did not reduce a single query because we are fetching queries as per object.

I don't think in this case we can go any further with object but instead maybe we can manipulate get_queryset which I'm unaware of any function that would help in this case.

I am not sure if draggable MPTT can help because I'm expecting something like this:

Screen Shot 2022-07-04 at 9 43 04 PM

matthiask commented 2 years ago

Ah yes. So, what you could do is extend the ChangeList class, override its get_results method (probably) and fetch all ancestors of all objects on the page in one go; I'd then store the ancestors in a private attribute (e.g. ad._cached_ancestors or something) and prefer this attribute to self.get_ancestors(...) in your category.

https://github.com/django-mptt/django-mptt/blob/7dbfcb67aaf630cf213a3ec7789f9e73056dceab/mptt/managers.py#L202 may help with this.

YDA93 commented 2 years ago

@matthiask

I did add below in get_queryset:

def get_queryset(self, request):
    qs = super().get_queryset(request).prefetch_related("images")
    self._cached_ancestors = (
        Category.objects.filter(id__in=qs.values("parent"))
        .select_related("parent")
        .get_ancestors(include_self=True)
    )
    return qs

Then I reference per object:

def category(self, obj):
    family = (
        self._cached_ancestors.get(id=obj.parent.id)
        .get_ancestors(include_self=True)
        .values_list("name", flat=True)
    )
    return " ➤ ".join(family)

I still get same duplicated queries.

Am I missing something?

YDA93 commented 2 years ago

@matthiask

Sorry, I feel so stupid.

What works is:

list_select_related = (
    "parent__parent__parent__parent__parent",
)

def category(self, obj):
    """Category Display"""
    family = []
    parent = obj.parent

    for __ in range(0, 4):
        if parent is None:
            break

        family.append(parent.name)
        parent = parent.parent

    return " ➤ ".join(family)

Now is back to 8 queries.