Closed osahon-okungbowa closed 4 years ago
Parsers for edoctae complete. Did the following:
Updated the octae crawler's "allowed_regex" to ensure all necessary urls can be crawled
Created 2 variant parsers in the 'edoctae.parsers' package to cater for octae pages
Stopped the parsing of document files as requested
Tested parser to ensure it runs with new scrapy pipelines, parses the pages and produced viable output
Parsers for edoctae complete. Did the following:
Updated the octae crawler's "allowed_regex" to ensure all necessary urls can be crawled
Created 2 variant parsers in the 'edoctae.parsers' package to cater for octae pages
Stopped the parsing of document files as requested
Tested parser to ensure it runs with new scrapy pipelines, parses the pages and produced viable output