Add a flag to crawl only the root website

[devavinothm]

Explanation of Changes

• Added stay_within_domain Parameter:

  This parameter is added to the __init__ method to control whether to crawl external links or not.

• Stored Root Domain:

  self.root_domain stores the root domain parsed from the root_url.

• Filtered Links:

  In the crawl method, before adding a link to self.crawl_set, check if the link’s domain matches the root domain when stay_within_domain is True.

I hope this solves the issue.

[indrajithi]

Also please check this

[indrajithi]

Closing due to inactivity. /cc @Mews @devavinothm