scrapy / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.
https://scrapy.org
BSD 3-Clause "New" or "Revised" License
51.16k stars 10.35k forks source link

Proxy tests don't use tests/keys/mitmproxy-ca.pem #6258

Closed wRAR closed 2 months ago

wRAR commented 2 months ago

We pass --certs tests/keys/mitmproxy-ca.pem, but --certs is for specifying website certs, not CA certs, so it's ignored and the user-wide ~/.mitmproxy/mitmproxy-ca.pem is used instead. We should either fix the way we specify the CA cert or (if that allows not using a CA cert at all) provide a website cert instead of a CA cert.

Laerte commented 2 months ago

Seems that we can use: https://docs.mitmproxy.org/stable/concepts-certificates/#using-a-custom-certificate-authority

wRAR commented 2 months ago

Yeah, that looks like the solution for the first option, though it notably takes a dir, not a file (but just passing tests/keys is unlikely to be problematic).