Closed arun477 closed 4 years ago
i have one classification code and i want to apply that code to each url during crawling and based on that it should decide whether to follow that url or omit that.
https://frontera.readthedocs.io/en/latest/topics/custom_crawling_strategy.html#writing-custom-crawling-strategy
i have one classification code and i want to apply that code to each url during crawling and based on that it should decide whether to follow that url or omit that.