search

rajatomar788 / pywebcopy

Locally saves webpages to your hard disk with images, css, js & links as is.
https://rajatomar788.github.io/pywebcopy/
Other
520 stars 105 forks source link

Bot being detected #129

Open wnsite opened 3 months ago

wnsite commented 3 months ago

My code:

save_webpage(url="https://minecraft.wiki", project_folder=os.getcwd()+"/test",project_name="my_site",bypass_robots=True,debug=True,open_in_browser=True,delay=None,threaded=False)

the code runs correctly, but the index.html file it downloads has one thing: 403: Forbidden minecraft.wiki is public and I am using bypass robots but it seems that it is still being detected.

rajatomar788 commented 3 months ago

Try changing the useragent. It should work.