Open mchehab opened 7 months ago
Analyze results for such query:
This is likely above my paygrade :sweat_smile: I can reproduce on the linuxtv instance but have yet to do so locally (and even then, I'm not certain how I'd address it yet...)
This is likely above my paygrade 😅 I can reproduce on the linuxtv instance but have yet to do so locally (and even then, I'm not certain how I'd address it yet...)
Yeah, this one seems tricky, and it generates 3 sub-queries that don't use indexes. Btw, if the query keeps having the blank fields there, like ?q=&series=&..
, the number of sub-queries without indexes rise to 4.
Funny enough, a query like ?archive=false
is fast. I suspect that the problem here is how patchwork/Django converts state=*
. into a query.
I mean, running:
analyze select state_id, count(*) from patchwork_patch where NOT archived GROUP BY state_id;
+------+-------------+-----------------+------+--------------------+--------------------+---------+-------+-------+----------+----------+------------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra |
+------+-------------+-----------------+------+--------------------+--------------------+---------+-------+-------+----------+----------+------------+--------------------------+
| 1 | SIMPLE | patchwork_patch | ref | patch_covering_idx | patch_covering_idx | 1 | const | 30608 | 44236.00 | 100.00 | 100.00 | Using where; Using index |
+------+-------------+-----------------+------+--------------------+--------------------+---------+-------+-------+----------+----------+------------+--------------------------+
1 row in set (0.047 sec)
is really fast. So, IMO, the problem is how patchwork currently handle "state=*", creating this really complex query, while something a lot simpler would produce the same result while using indexes.
Looking at the code, it sounds it comes from here:
class PatchQuerySet(models.query.QuerySet):
def with_tag_counts(self, project=None):
if project and not project.use_tags:
return self
# We need the project's use_tags field loaded for Project.tags().
# Using prefetch_related means we'll share the one instance of
# Project, and share the project.tags cache between all patch.project
# references.
qs = self.prefetch_related('project')
select = OrderedDict()
select_params = []
# All projects have the same tags, so we're good to go here
if project:
tags = project.tags
else:
tags = Tag.objects.all()
for tag in tags:
select[tag.attr_name] = (
"coalesce("
"(SELECT count FROM patchwork_patchtag"
" WHERE patchwork_patchtag.patch_id=patchwork_patch.id"
" AND patchwork_patchtag.tag_id=%s), 0)"
)
select_params.append(tag.id)
return qs.extra(select=select, select_params=select_params)
No idea why it is trying to count patches there via such complex query, instead of just doing the query.
This query:
state=*&archive=true
Is producing a complex sql statement that it is not using indexes and are taking a long time to complete:As reported by mysql slow log, it takes more than 1:30 mins to complete:
Issue noticed at https://patchwork.linuxtv.org.