Closed danyaljj closed 7 months ago
Hey, This issue can arise either if you are using different subdomains of the given website or if you don't have the scraping permission for the site.
So try the 'https://www.google.com/' instead of 'https://google.com' that you used. Also try the bypass_robots option equals True.
When I try the script for famed websites such as Google, I am getting error:
Is this expected? is there a way to bypass these?
Here is my script: