python-discord / site

pythondiscord.com - A Django and Bulma web application.
https://pythondiscord.com
MIT License
638 stars 127 forks source link

Evaluate updating homepage repositories outside of the view #1285

Open jchristgit opened 3 months ago

jchristgit commented 3 months ago

Presently, when the homepage view is opened, it will sometimes fetch new repository metadata from GitHub and update this information in the database. When data is newly fetched, this is pretty slow. The question is: Do we want to run it separately?

Benefits of running it separately:

Drawbacks of running it separately:

jchristgit commented 3 months ago

One alternative approach that might be a bit hacky but would likely be the easiest to implement: we could start a background thread using threading that would do the updating after we've returned the response. I would strongly favor not using a job queue such as Celery since I've had very bad experiences with it, and for this single use case it would feel kind of overblown. Additionally, implementing it would require a lot of additional monitoring, deployments, and introduce a lot more components that could fail. So I'm really not in favor of it.

jchristgit commented 3 months ago

Maybe a bit hacky as well, but another option that we have would be to use Django's asynchronous support to create an async task that should run after the request. I think to make that fully work we would have to run under an ASGI server though, and transform all the views to properly run with it.