Include fetching in scrapers

lorey / mlscraper

🤖 Scrape data from HTML websites automatically by just providing examples

1.31k stars 89 forks source link

Closed lorey closed 2 years ago

lorey commented 4 years ago

Scrapers should be able to deal with urls, HTML, and parsed DOMs (even requests response objects?) to enable for flexible library usage.

scrapers input HTML, DOM, and responses, e.g. via scrape_soup, scrape_html, scrape_url
examples can be created via static methods, e.g. via SingleItemPageSample.from_soup, etc.

lorey commented 2 years ago

Currently omitted to avoid requests dependency. Could be relevant if asked for frequently.