search

SJDunkelman / read-all-about-it

🗞 Scrape headline & article text data from major US/UK Newspaper sources for use in NLP projects
GNU General Public License v3.0
1 stars 1 forks source link

NY Post #2

Closed SJDunkelman closed 4 years ago

SJDunkelman commented 4 years ago

Base URL: www.nypost.com/news/

Need to interact with 'Load More' button to get later results. Need to use Selenium + Chrome and the XPath '/html/body/div[1]/div[5]/div[2]/div/div[1]/div/div[1]/div[2]/div/div[6]/div/a' to scrape URLs.