如何设置https代理

code4craft / webmagic

A scalable web crawler framework for Java.

http://webmagic.io/

Apache License 2.0

11.44k stars 4.18k forks source link

如何设置https代理 #650

Open jayyuz opened 7 years ago

jayyuz commented 7 years ago

请问作者，如何实现https代理？我需要怕的网站都是https的。最近爬的次数太多，被封了。

yitao2 commented 7 years ago

Proxy proxies[] = HttpUtils.getProxyIPs(); // 得到线程池 HttpClientDownloader httpClientDownloader = new HttpClientDownloader(); httpClientDownloader.setProxyProvider(SimpleProxyProvider.from(proxies));

Spider.setDownloader(httpClientDownloader)

jayyuz commented 6 years ago

蟹蟹