Add `internal_links_only` and `external_links_only` options

indrajithi / tiny-web-crawler

A simple and easy to use web crawler for Python

MIT License

55 stars 11 forks source link

Add `internal_links_only` and `external_links_only` options #40

Closed Mews closed 1 week ago

Mews commented 1 week ago

Closes #11

Changes Added internal_links_only and external_links_only options to skip links that match or dont match the root url netloc. Also added two test cases for these two options.