Open Sovietaced opened 5 hours ago
Hmm. Looking at the code now it looks like the job_run
table has an index on job id. The job
table primary key is a job_id so that should be a native index.
I do notice that when the DB pruner runs it pushes the CPU utilization of the DB instance over the 2 CPU max so perhaps the DB pruning is causing some CPU contention for our AWS RDS instance and just slowing down all queries.
@d80tb7 who is best to assist with Lookout?
Describe the bug We have enabled the lookout DB pruner and it takes the DB pruning logic quite a lot of time to delete rows. We're currently running through a backlog of old jobs on production and it looks like it will take ~4 days of DB pruning time to get through it.
I haven't looked into the query planner yet but I'm assuming there is a lack of indexes for some of these queries and a linear scan is being performed on several million rows. We're running on a db.r7g.large AWS RDS instance.