httr2 cache error when building a package website

[maelle]

I got an error while running pkgdown::build_site() for rotemplate. I'm reporting this here in case it's not just due to a weird setup of mine.

I ran the same function last week and it worked well.

Today I got

#> Error: ! in callr subprocess.
#> Caused by error in `readRDS(path)`:
#> ! error reading from connection

This comes from

pkg <- "rotemplate"  
bioc_url <- paste0("https://www.bioconductor.org/packages/", pkg)
req <- httr2::request(bioc_url)  
req <- pkgdown:::req_pkgdown_cache(req)
req <- httr2::req_error(req, function(resp) FALSE)
req <- httr2::req_retry(req, max_tries = 3)
resp <- httr2::req_perform(req)
#> Error in readRDS(path): error reading from connection

^{Created on 2024-07-05 with reprex v2.1.0}

Manually deleting the cache fixed it (I ran the code twice so that the cache was both created and used). However I now wonder, how could the cache get invalid? Should pkgdown retry without using the cache? Or can we hope it won't happen again?

[olivroy]

got the same error while running pkgdown tests and had to do the same manual resolution https://github.com/r-lib/pkgdown/pull/2676#issuecomment-2195455780 (R 4.3.2)

[jayhesselberth]

FWIW I have also seen this recently and it went away on its own (maybe when I upgraded to R 4.4.1?).

[hadley]

I've seen this too 😬 It must be a bug in httr2 but I'm not sure what would lead it to write a corrupt RDS file.

[hadley]

Are y'all running the pkgdown tests or just pkgdown itself?

[olivroy]

For me, I encountered this while running pkgdown tests, not while building pkgdown sites (for other packages)

[hadley]

Hmmmm I wonder if this might be a race condition from running the tests in parallel.

[maelle]

I was building a website.

[maelle]

hey I just created an invalid rds while rerunning my reprex, exciting! the rds file is called 96e4e0c00a1e68ed3271453bb57d5a0b.rds but Ubuntu recognizes it as a text file. When I open it with a text editor, it features one empty line (its size is 0).

[maelle]

what does an empty httr2 response look like?

[maelle]

could that file be created by cache_get() (that touches the file) but then not written into because the status code isn't 200 or so?

[hadley]

Oooh yeah, maybe it should only touch if the file already exists!

...

Hmmmm but cache_get() should only get called if it exists?

[maelle]

oh right.

I wish the debug argument of req_cache() were responsive to an option (i want to try using it) (update: I did not catch the bug while using debug)