redapple
commented
7 years ago By default, scrapy does not check remote server certificates. So I dont think installing Crawlera's certificate is needed with scrapy-crawlera. Another thing, I believe scrapy-crawlera still use the http:// proxy without CONNECT (which should be changed eventually)
Le 25 mars 2017 15:49, "Pablo Hoffman" notifications@github.com a écrit :
I've heard many Crawlera users complaining about installing the certificate (to make https work with Crawlera) and I was thinking that, since the Crawlera middleware is meant to make things easier on the client side, it could automatically download and install the certificate if it's not present, to reduce another step in the "crawlera setup process". Installing the certificate is not a major task, but also not a trivial one, and it varies per platform. But most importantly, it can be automated in things like the Crawlera middleware.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/scrapy-plugins/scrapy-crawlera/issues/41, or mute the thread https://github.com/notifications/unsubscribe-auth/AA2GGIo7T78XWdFQJcWjLbU5FnHDIV7xks5rpSl5gaJpZM4MpGQC .
pablohoffman
I've heard many Crawlera users complaining about installing the certificate (to make https work with Crawlera) and I was thinking that, since the Crawlera middleware is meant to make things easier on the client side, it could automatically download and install the certificate if it's not present, to reduce another step in the "crawlera setup process". Installing the certificate is not a major task, but also not a trivial one, and it varies per platform. But most importantly, it can be automated in things like the Crawlera middleware.