Open MihirUpasani opened 6 years ago
I disabled certificate verification as it was failing in a number of cases. There was no point having my scrapes fail because of some poorly configured SSL certificate or similar error.
The scraper uses HTTP instead of HTTPS anyway. Doing full SSL of the scraper would require you to:
amazon_base
url.request.get
's to not use verify=False
Again, I don't see this as an issue as no user information is passed. The Amazon API is a separate module to the scraper.
/home/####/.local/lib/python2.7/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning)
This is the warning i get on trying this :
from amazon_scraper import AmazonScraper amzn = AmazonScraper(I've passed correct arguments) rs = amzn.reviews(ItemId='B0734X8GW5') for r in rs.ids: rvn = amzn.review(Id=r) print (rvn.id) print (rvn.text)