search

fedora-copr / copr

RPM build system - upstream for https://copr.fedorainfracloud.org/
115 stars 61 forks source link

prune-dist-git causes huge Frontend API traffic #3210

Open praiskup opened 6 months ago

praiskup commented 6 months ago

Optimize-out the large API (daily in Fedora Copr) caused by: https://github.com/fedora-copr/copr/blob/main/dist-git/run/prune-dist-git.py

Currently it causes 30k API calls daily.

praiskup commented 6 months ago

Proposal: Let's have a /backend/active-dirs/ route (can require FE token authentication to avoid DoSing by non-distgit users) providing a list of all CoprDirs (that's a relatively cheap query, <= 50k directories now, <= 5MB of data).

praiskup commented 6 months ago

Currently 28656 API calls:

# wc -l /var/log/copr-dist-git/api-calls-2024-04-09.log 
28656 /var/log/copr-dist-git/api-calls-2024-04-09.log