HTTP error 544 when trying to read_html()

tidyverse / rvest

Simple web scraping for R

https://rvest.tidyverse.org

Other

1.49k stars 343 forks source link

HTTP error 544 when trying to read_html() #178

Closed sillasgonzaga closed 8 years ago

sillasgonzaga commented 8 years ago

X-post from this SO question.

I want to use rvest to scrape this page. However, my code returns a HTTP error even though the page is working fine.

myurl <- "http://rj.olx.com.br/rio-de-janeiro-e-regiao/imoveis/aluguel/aluguel-de-quartos?o=2"
x <- myurl %>% read_html()
Error in open.connection(x, "rb") : HTTP error 544.

What does this error mean? How can I avoid it? Am I getting blocked by the website?

sillasgonzaga commented 8 years ago

Actually I solved the problem using the solution described here.