Check universality for body crawling investigate - Githubissues

bgabor99 / News_crawler

0 stars 0 forks source link

Check universality for body crawling investigate #17

Closed bgabor99 closed 1 year ago

bgabor99 commented 1 year ago

Can be run on several domains? with the same script?
Can crawl every page in the document?

bgabor99 commented 1 year ago

Rebased spiders into one per domain
- Use LxmlLinkExtractor
Pipeline is rebased for this