Create WebCrawler/Scraper Class - Githubissues

graisezr / Anthology-Maker

generate a list of poems based on ur preferred theme and category

0 stars 0 forks source link

Create WebCrawler/Scraper Class #1

Open mifi555 opened 1 year ago

mifi555 commented 1 year ago

Create WebCrawler/Scraping class
Find out a way to dynamically scrape a website
Use Selenium
ChromeDriver: move chromedriver to /usr/local/bin https://www.swtestacademy.com/install-chrome-driver-on-mac/

sudo mv /Users/milanfilo/Desktop/chromedriver_mac64/chromedriver /usr/local/bin/

Download SL4J library: https://www.tutorialspoint.com/slf4j/slf4j_environment_setup.htm

Find out if the website was scrapable: https://poets.org/robots.txt

mifi555 commented 1 year ago

WebScraper complete