Closed captjt closed 7 years ago
Creating a modular way to do custom scraping through settings is a bigger lift will have to have specific ways we want to drill into html nodes.
Bi-product will be multiple package files and an API to use ... super simple and super powerful
Premise of Refactor
Prototype Crawler (ETL) is working but because we want to have an easy way to reproduce these custom crawlers and scrapers ...
... we need to modularize the code to be reusable packages that we can customize to our wishes