search

ubccr / coldfront

HPC Resource Allocation System
https://coldfront.readthedocs.io
GNU General Public License v3.0
96 stars 76 forks source link

Reduce admin page load times #461

Open claire-peters opened 1 year ago

claire-peters commented 1 year ago

We’re facing an issue in the FASRC fork of Coldfront that is likely shared by the UBCCR repo. The admin interface has always been a little laggy, but with the growth of our database, load times for some pages have gotten unreasonably long. We’ve recorded an Allocation that has around 30 AllocationUsers taking minutes to load, for instance.

I used Django Debug Toolbar to investigate further. One thing that stood out in the results was the number of duplicate queries the page with the minutes-long load time produced in the course of loading. Based on the number of times the resource_resource query is repeated, for instance, it seems like taking the advice recommended here would reduce Allocation admin page load time. There may be similar improvements to make on this and other pages, but I thought I’d start the conversation there.

Here are some screenshots of the Django Debug Toolbar output. Rendering in debug mode apparently adds to load time, but load time is still significant outside debug mode.

IMG_0018

IMG_0020

Here’s a screenshot of some of the queries. Altogether there were 476 queries (466 similar and 396 duplicates), which took a total of 23995.46 ms. Note the 134 resource_resource and 66 allocation_allocation query duplications:

IMG_0016

Happy to provide any additional information that might be helpful and/or develop a solution.

dsajdak commented 1 year ago

@claire-peters thanks very much for this information. You're right that we don't want all those duplicate queries so we'll be looking into this to see why this is. I was surprised to read that you've always experienced some lag with your admin interface as I have never had that problem. We've been running CF for years now and the database has about 775 projects, 2400+ allocations, 300 resources, and 6000 users. What is the size of your database and what type of database are you running? Thanks!

claire-peters commented 1 year ago

@dsajdak Our database is in fact much smaller - we have around 140 Projects, 300 Allocations, 5200 Users, and 5 Resources. We run MariaDB in a Docker container that's networked to a separate container that runs the Coldfront application. The Admin app appears to be the only one that experiences these kinds of delays.

The primary factor in the Allocation admin page lag seems to be how many AllocationUsers a given Allocation has - for Allocations with one user the load time is a few seconds at most, but Allocations with many AllocationUsers will take much longer.

rg663 commented 1 year ago

Hi @claire-peters! Thank you for your patience as we investigate why this is. We suspect it has to do with the __str__ methods in the allocation/models.py file adding many queries, but we're looking into other possible reasons as well. Did your version of ColdFront always have this lag issue, or was this after a recent change made on our end?