I want the feature of crawling the links of website first and then scrape the crawled links
please add the all features make this roboust scraper so it can be used for other projects too
This is the sample of Python scaper
def parse_html(html):
Adjust this selector to match the reviews on Amazon's page
reviews = html.css("div[data-hook='review']")
for review in reviews:
yield Review(
title=extract(review, "a[data-hook='review-title']", output="text"),
# Implement the extraction logic for helpful and body
)
The Website https://www.amazon.com
I want the feature of crawling the links of website first and then scrape the crawled links
please add the all features make this roboust scraper so it can be used for other projects too This is the sample of Python scaper
def parse_html(html):
Adjust this selector to match the reviews on Amazon's page