adamcharnock / django-hordak

Double entry accounting in Django
http://django-hordak.readthedocs.io
MIT License
231 stars 55 forks source link

Slow TransactionAdmin list view #52

Closed PetrDlouhy closed 2 weeks ago

PetrDlouhy commented 3 years ago

The TransactionAdmin can be very slow due to vast number of DB queries. I tried to fix this with partial success - I reduced number of queries from ~900 to ~400 in my test case by this admin override:

@admin.register(models.Transaction)
class TransactionAdmin(hordak_admin.TransactionAdmin):
    def debited_accounts(self, obj):
        return ", ".join([str(leg.account) for leg in obj.debit_legs]) or None

    def total_amount(self, obj):
        return obj.total_amount

    def credited_accounts(self, obj):
        return ", ".join([str(leg.account) for leg in obj.credit_legs]) or None

    def get_queryset(self, *args, **kwargs):
        return super().get_queryset(*args, **kwargs).prefetch_related(
             Prefetch('legs', queryset=models.Leg.objects.filter(amount__gt=0).select_related("account"), to_attr="debit_legs"),
             Prefetch('legs', queryset=models.Leg.objects.filter(amount__lt=0).select_related("account"), to_attr="credit_legs"),
        ).annotate(
            total_amount=Sum("legs__amount", filter=Q(legs__amount__gt=0)),
        )

The code could be probably better directly implemented in LegManager to make use of credits() and debits(), but anyway - I was not able to reduce number of queries to constant number and implement the prefetches in the manager.

adamcharnock commented 2 weeks ago

I'm considering create another view for this, much like in #120.

adamcharnock commented 2 weeks ago

I've just added a test for this and it seems to now be executing in a fixed number of queries (8, to be precise). So I am suspecting that this is now fixed.

In fact, fixed to the extent that I suspect there is no longer a need for sql_util.


Update: See #127

adamcharnock commented 2 weeks ago

Closing this as it seems to be resolved, but definitely shout if it is still an issue