search

tidyverse / rvest

Simple web scraping for R
https://rvest.tidyverse.org
Other
1.49k stars 341 forks source link

Error with Scrapping Multiple Pages with ForLoop #367

Closed emirizatrashid closed 1 year ago

emirizatrashid commented 1 year ago

Hi! I used a for loop to scrape multiple pages and I received an occasional error:

Error in readBin(3L, "raw", 65536L) : transfer closed with outstanding read data remaining

I tried running the code again and it worked, but sometimes run into the same error after 10-20 pages. The longest stretch was running a loop for 600 pages (I thought the problem was over at this point) before encountering it again.

hadley commented 1 year ago

Unfortunately there's not much we can do without a reprex.