osu-tournament-rating / otr-api

API powering osu! Tournament Rating
https://otr.stagec.xyz/
GNU General Public License v3.0
8 stars 5 forks source link

Investigate excess audits #546

Open hburn7 opened 1 day ago

hburn7 commented 1 day ago

When I last checked the database, I think we had around ~60M audits for Games - some really big number.

Need to investigate how audits are being generated and verify that it's behaving as expected. The database continues to fill up to capacity and will undoubtedly lead to a critical failure in prod if we fail to properly address runaway audits.

myssto commented 1 day ago

What is likely the case is that there are millions of audits per each entity type that are simply just updates to the LastProcessingDate. Each time a processor is ran on any entity, the entity's LastProcessingDate is updated. Each time a tournament processor is ran, there is a save changes call made on completion. This will cause roughly ~4 audits to be made for every dependent auditable entity, every time a tournament is pulled for processing :P

It's something I hadn't thought about until now but I'll start thinking on a solution