Removal of content from a config takes time to be computed

[Popolechien]

Creating a fairly large configuration and trying to remove zim files from it, it takes several seconds for it to be taken into account in the config size counter (during which time other operations such as Change tab or Update config are not responding). The zim file disappears from the content list almost immediately, however.

[rgaudin]

Thank you for this. I squashed multiple bugs behind this innocent-looking remove button

• we were downloading selected language's catalog each time we removed an item from the selection (to rebuild the catalog table from scratch). Returning the English catalog is inexpensive but still a useless request + JSON encoding +GZIP + I/O.
• introduction of pictures in edit seemed harmless using async but on large catalog like English it meant approx. 1,000 requests to library.kiwix.org. Browsers have a limit on the number of requests they can make (900 for firefox) so other requests have to wait for previous ones to complete before being sent. This accounted for most of the delay. We're now using loading="lazy"
• bug around event handling having the UI make an exponential number of request to the language catalog resulting in a bottleneck: so that's 1,000 + growing API calls ; all that on each removal 😅
• bottleneck could (not confirmed) also happen on manager side with those growing number of requests. I increased the number of processes and make them static (see following)
• catalog refresh can take a few seconds to complete: it downloads the whole OPDS, parses it and stores and id-indexed one and a lang-sorted index. refresh is once per hour but it you hit it, it's not comfortable ; especially if it delays other stuff. refresh is now independent of request (we return cache then refresh)
• uwsgi's processes most likely (haven't checked internals) each keep a copy of the catalog. So refresh was not once per hour for everybody but once per hour per worker. So a single user could hit it twice within an hour ; especially with the dynamic processes that we were using. there's also a race condition that have us refresh twice on process start (but I haven't fixed that one). I've also added uwsgitop to the image so we can have better insights at what happens on prod.