search

r-universe-org / help

Support and bug tracker for R-universe
https://docs.r-universe.dev/
8 stars 2 forks source link

Investigate overloads #307

Open jeroen opened 8 months ago

jeroen commented 8 months ago

When we triggered like 500 rebuilds for https://cran.r-universe.dev the server gets overloaded, but I am not sure why. Maybe there is a bug with generating the packages files but that should be cacheble. But maybe pak is cicrumventing that somehow.

Could also be that the total load was just too much especially because pak downloads all deps concurrently, but I did not seem like that was the problem.

Or maybe it is already fixed with: https://github.com/r-universe-org/help/issues/305

The workaround we put in place: https://github.com/r-universe/workflows/commit/12e106344109f686bba2cf045c5a354deb940ad0. So to reproduce, revert the workaround and trigger a batch of rebuilds for the cran universe.

jeroen commented 8 months ago

I found that available.packages this does not seem to hit the cache, even though it should.

df=available.packages(repos = 'https://cran.r-universe.dev', ignore_repo_cache=T)

Because available.pakages() uses download.file(... cacheOK = FALSE) which sets "Pragma: no-cache" but nginx should not honor that by default, see: https://www.nginx.com/blog/nginx-caching-guide/