search

fedora-infra / bodhi

Bodhi is a web-system that facilitates the process of publishing updates for a Fedora-based software distribution.
https://bodhi.fedoraproject.org
GNU General Public License v2.0
151 stars 189 forks source link

Query timeouts #5646

Open mattiaverga opened 2 months ago

mattiaverga commented 2 months ago

When searching updates with a high number of results the query hits the timeout. Maybe this was the culprit of recent server restarts. To reproduce, go to the updates page and search for "plasma" using the default settings.

I think we need to refine the query defaults to be much more limited. BTW, the defaults search in "all pending, all current" releases, which means include ELN which is a never ending release which will always grew the list of results.

mattiaverga commented 2 months ago

From further investigation, this is indeed what is causing server hiccups, at least before we added a query timeout setting.

The problem here (which I can reproduce in the local dev environment) is that loading an Update object causes a waterfall of loading all its builds, all comments, all feedback, etc. So when a user search for a package which belongs to an update with a lot of builds, this causes a lot of sql traffic - for example, searching for kpmcore with the default settings in the webUI form causes ~10800 queries and takes 25 seconds in my dev environment...

I tried to play with sqlalchemy options and refining queries, but I only got marginal improvements. No idea how to fix this...

mattiaverga commented 1 month ago

A little bit info here: when opening the update list in a web page, it triggers the __json__() method for every update object listed. I can't understand why, since the html renderer should only return the plain object (I think). This triggers the well known json serialization nightmare when the update list is composed by updates with a lot of builds.