Problem with InsecureRequest

adamlwgriffiths / amazon_scraper

Provides content not accessible through the standard Amazon API

Other

234 stars 60 forks source link

Problem with InsecureRequest #39

Open MihirUpasani opened 6 years ago

MihirUpasani commented 6 years ago

/home/####/.local/lib/python2.7/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning)

This is the warning i get on trying this :

from amazon_scraper import AmazonScraper amzn = AmazonScraper(I've passed correct arguments) rs = amzn.reviews(ItemId='B0734X8GW5') for r in rs.ids: rvn = amzn.review(Id=r) print (rvn.id) print (rvn.text)

adamlwgriffiths commented 6 years ago

I disabled certificate verification as it was failing in a number of cases. There was no point having my scrapes fail because of some poorly configured SSL certificate or similar error.

The scraper uses HTTP instead of HTTPS anyway. Doing full SSL of the scraper would require you to:

change the amazon_base url.
change all request.get's to not use verify=False

Again, I don't see this as an issue as no user information is passed. The Amazon API is a separate module to the scraper.