search

tidyverse / rvest

Simple web scraping for R
https://rvest.tidyverse.org
Other
1.49k stars 342 forks source link

Scraping from javascript websites #183

Closed higgi13425 closed 5 years ago

higgi13425 commented 7 years ago

An example is http://kff.org/other/state-indicator/total-residents/?currentTimeframe=0 simple table with data (one of many useful pages they share publicly), but not accessible via rvest.

It appears that there is an obscure workaround : https://www.datacamp.com/community/tutorials/scraping-javascript-generated-data-with-r#gs.7KHOcV0 but it is not very accessible to non-coders.

Any way to make this a under-the-hood capability of rvest?

nick-ulle commented 7 years ago

@higgi13425 An all-R alternative to the workaround you've found is to use the RSelenium package, which allows you to control a web browser (or PhantomJS) from R. The RSelenium interface is simple but installing Selenium/PhantomJS can sometimes be difficult for inexperienced users.